gee-community / geetools Goto Github PK
View Code? Open in Web Editor NEWA collection of tools to work with Google Earth Engine Python API
Home Page: https://geetools.readthedocs.io/en/stable/
License: MIT License
A collection of tools to work with Google Earth Engine Python API
Home Page: https://geetools.readthedocs.io/en/stable/
License: MIT License
Hi! Used to work in the past, but when I batch export sentinel imagery with only '{:system_date:}' as the name pattern the imagery all exports as ":system_date:" rather than its respective name e.g. "2017-01-12"
Hi Rodrigo -
(You helped me get gee_tools set up a few years ago! Thank you!)
I'm trying to make and download cloud-free timelapse videos of specific regions in Greenland, with the image date on each scene. I "think" I have it going in the code editor (ugly, I know!)
https://code.earthengine.google.com/?accept_repo=users/leighastearns/shared
I'd like to get it working in Python, since that's a lot easier and faster. I can get a basic video downloaded with batch.Export.video.to.drive, but my questions:
These seem like they should be such easy tasks, but I seem to be going in circles parsing them.
thanks for any help,
leigh
HI!,
I would like to download an Image collection (more than 80 GB) into my personal computer, because google drive doesn't offer a lot of storage.
Is that possible? and if yes, How?
Thank you
In geetools/ui/maptool.py the option to add opacity for addLayer does not seem to work. I think it is because in row 103 of that script you did not include opacity=params['opacity'] when creating the folium.TileLayer ....
Dear Rodrigo,
Thank you very much for your gee_tools library for python. I am trying to utilize it into my study.
It is okay to map with s2ESA (cloud_mask.sentinel2) without error, however when I map with s2holl (cloud_mask.hollstein_S2), my app produce this error: Invalid argument specified for ee.List(): hollstein
Could you please guide me how to fix? Or did I do something wrong?
Thank you!
ndvi = ee.ImageCollection('MODIS/006/MOD13A2').filterDate('2009-01-01','2017-12-31').select('NDVI')
ndvi_mon = ndvi.filter(ee.Filter.calendarRange(2009,2017,'year')).filter(ee.Filter.calendarRange(6,7,'month'))
from geetools import tools
tasks = tools.col2drive(ndvi_mon, 'NDVI0607', scale=1000)
try n° 0 ERROR: name 'TYPES' is not defined
try n° 1 ERROR: name 'TYPES' is not defined
try n° 2 ERROR: name 'TYPES' is not defined
try n° 3 ERROR: name 'TYPES' is not defined
try n° 4 ERROR: name 'TYPES' is not defined
Even though I added (dataType='float'), it is still the same error.
The mosaicSameDay function in imagecollection.py is pretty handy, but it fails to maintain the proper datatypes in the output mosaic image bands. Instead, each band becomes a Float (at least when I tried with Sentinel 2). Would it be possible to map the band datatype from the original image to the mosaiced one?
The ImageCollection Export to Drive function seems useful. I think it would be useful to add similar methods for other export destinations. My particular use case requires ImageCollection Export to Cloud Storage, which doesn't seem to be implemented here or on base EE. There may be other export methods missing from EE that would be useful here as well.
(Also, thanks for the neat library!)
Hi Rodrigo -
Thanks again for your earlier help. Have you done any work with MODIS?
I realize that using python to batch download images means that MODIS gets downloaded in its original (annoying) sinusoidal projection. My attempts with gdalwarp and gdal_translate after downloading the images give me a syntax error:
vel='/Users/leigh/Google Drive/GEE/KGL/MODQ/MOD09GQ_005_2017_03_26.tif'
gdalwarp -s_srs '+proj=sinu +lon_0=0 +x_0=0 +y_0=0 +a=6371007.181 +b=6371007.181 +units=m +no_defs' vel vel_out.tif
Is there a way to reproject it before downloading? Thanks, and sorry to bug you again!
Documentation saids that the function will return a list of tasks
:return: list of tasks
:rtype: list
but there is not return at the end of the function.
As the list of tasks is already created, it is just necessary to add :
return tasklist
at the end of the function
It works fine in python 3.5, but in python 3.4 when doing from geetools import ui
it throws an error:
ImportError Traceback (most recent call last)
in ()
----> 1 from geetools.ui import ipymap~/Projects/eds_vietnam/env/eds_vietnam/lib/python3.4/site-packages/geetools/ui/init.py in ()
4 import threading
5 import pprint
----> 6 from . import chart, imagestrip, ipymap, ipytools, maptool, dispatcher
7
8 from IPython.display import display~/Projects/eds_vietnam/env/eds_vietnam/lib/python3.4/site-packages/geetools/ui/ipymap.py in ()
14 from .maptool import inverseCoordinates, getImageTile, getGeojsonTile,
15 getBounds, getZoom, featurePropertiesOutput
---> 16 from . import maptool, ipytools
17 import threading
18 from copy import copy~/Projects/eds_vietnam/env/eds_vietnam/lib/python3.4/site-packages/geetools/ui/ipytools.py in ()
7
8 from .. import batch
----> 9 from .dispatcher import widgetDispatcher
10
11 # imports for async widgets~/Projects/eds_vietnam/env/eds_vietnam/lib/python3.4/site-packages/geetools/ui/dispatcher.py in ()
3 import ee
4 from ipywidgets import *
----> 5 from . import ipytools
6
7 def belongToEE(eeobject):ImportError: cannot import name 'ipytools'
python 3.6 is not supported
Hi folks,
I am trying to upload a table in my asset, however, I couldn´t do it. I selected the dbf, shp, and shx files as described the GEE. after minutes of running the process, I received the warning message: Error: Unable to parse the specified file.
Can anyone help me?
Hi Rodrigo!
It seems very interesting the library you've written. I'm looking for more information on how to use the cloud mask functionality. Could you give me an example?
Hello!
I have some dataset. I download pictures from this dataset.
I use gee-tools to export collections of images.
However, some downloaded files have values of all pixels NaN or have names of this kind and also have values of all pixels NaN
Altaiskii_krai__Nikolaevka_2018-10-15_S2TOA-0000000000-0000000000.tif
Altaiskii_krai__Nikolaevka_2018-10-15_S2TOA-0000000000-0000006912.tif
Altaiskii_krai__Nikolaevka_2018-10-15_S2TOA-0000006912-0000000000.tif
Altaiskii_krai__Nikolaevka_2018-10-15_S2TOA-0000006912-0000006912.tif
I learned to recognize such "zero" pictures by their volume. But I want to understand the reason for this. I want to remove such pictures from export.
Here's my code.
sat = geedatasets.sentinel.Sentinel2SR()
s2toa = geedatasets.fromShortName('S2TOA')
##collection = s2toa.collection().filterBounds(point)
#collection = collection.filterDate(start, end)
#collection = collection.filterMetadata("CLOUD_COVERAGE_ASSESSMENT", 'less_than', 20.0)
visParam = {'bands':['B4', 'B3', 'B2'], 'min':0, 'max':5000}
vis = sat.visualization('TrueColor')#('NSR')
counImages = 0
def get_dates(col):
dates = ee.List(col.toList(col.size()).map(lambda img: ee.Image(img).date().format()))
return dates
TotalSizeImageCollection = ee.Number(0)
sel = 2
for q in arrNump:
if(validate(q[3]) == True):
if (isinstance(q[6], str)):
q[6] = transliterate(q[6])
if (isinstance(q[7], str)):
q[7] = transliterate(q[7])
listPoint = ee.List([float(q[6])]).add(float(q[7]))
#listPoint = ee.List([55.958727]).add(54.735147)
p = ee.Geometry.Point(listPoint)
site = (p.buffer(30000)).bounds()
def clp(img):
return img.clip(site)
startDate = ee.Date(formatterdate(q[3])).advance(-10,'day')
if(isinstance(q[2], int)):
endDate = ee.Date(startDate.advance(10,'day'))
else:
if(validate(q[2]) == True):
endDate = ee.Date(formatterdate(q[2])).advance(10,'day')
col = sat.collection().filterBounds(site).filterDate(startDate, endDate).filterMetadata("CLOUD_COVERAGE_ASSESSMENT", 'less_than', 30.0)
countMos = 0
if(col.size().getInfo()>0):
countMos = col.size().getInfo()
global counImages
counImages = counImages + countMos
for i in range(col.size().getInfo()):
mosaics = tools.imagecollection.mosaicSameDay(col)
mosaics = mosaics.map(clp)
region = transliterate(q[0])
city = transliterate(q[1])
scale = 10
name_pattern = region + '_'+ city + '_' + '{system_date}_{sat}' #{system_date}_{sat}_{WRS_PATH:%d}-{WRS_ROW:%d}
date_pattern = 'Y-MM-dd' # dd: day, MMM: month (JAN), y: year
folder = region
data_type = 'float'
extra = dict(sat='S2TOA')
region = site
print(sel, q[3], q[2], q[1], q[0])
if(sel>110):
tasks = geetools.batch.Export.imagecollection.toDrive(collection=mosaics, folder=folder, region=region, namePattern=name_pattern, scale=scale, dataType=data_type, datePattern=date_pattern, extra=extra, verbose=True, maxPixels=int(1e13))
sel = sel + 1
if(sel>120):
break
print(sel)
When I tryng to install the package i get out this issue:
Collecting geetools
Using cached https://files.pythonhosted.org/packages/1d/57/f4d913a480480d44cfa94c78911b935dd40df2a2135be85d529741b21b91/geetools-0.1.1.tar.gz
Command "python setup.py egg_info" failed with error code 1 in C:\Users\xxxxx\AppData\Local\Temp\pip-install-ajxqlt3t\geetools
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\xxxxxx\AppData\Local\Temp\pip-install-ajxqlt3t\geetools\setup.py", line 5, in
from geetools import version
File "C:\Users\xxxxxxx\Local\Temp\pip-install-ajxqlt3t\geetools\geetools_init_.py", line 20, in
from .ui.ipymap import Map
File "C:\Users\GE899~1.DEL\AppData\Local\Temp\pip-install-ajxqlt3t\geetools\geetools\ui_init_.py", line 12
def eprint(eeobject, indent=2, notebook=False, async=False):
^
SyntaxError: invalid syntax
I am not sure if this is a bug, but I'll report it and you can hopefully determine. I am using the python API and gee_tools. I have shared my shapefile asset so hopefully you can just plug this in.
I am computing the area of a Landsat image within a polygon, and then computing it again after using the mosaicSameDay tool. The image I'm using completely covers the polygon, so one would expect the area to be the same before or after running the tool. However, this is not the case and the areas are substantially different.
import ee
from geetools.tools import imagecollection as ictools
# function to compute overlap area between polygon and image
def valid_area(img):
"""
Returns a feature containing the total valid area (as defined by the
append_vo_band function) within an image. Requires a 'valid_obs' band to
be present in an image.
"""
# Convert image to binary
bin_im = ee.Image(img.select('B1'))
bin_im = bin_im.where(bin_im.neq(0), 1)
bin_im = bin_im.where(bin_im.neq(1), 0)
# Convert valid observation band to pixel areas band (no valid obs are still zeros)
pix_areas = ee.Image.multiply(bin_im, ee.Image.pixelArea())
# Number of pixels in valid region
validarea = pix_areas.reduceRegion(
reducer = ee.Reducer.sum(),
geometry = feat.geometry(),
scale = scale,
maxPixels = 1e12,
bestEffort = False)
return ee.Feature(None, {'valid_area':validarea.get('B1')})
# Set things up
ee.Initialize()
start_date = '2010-01-01'
end_date = '2019-01-01'
scale = 30 # The scale to do the analysis
n_images = 10 # Maximum number of images to fetch--used for testing
provinces = ee.FeatureCollection('users/jonschwenk/Ecuador_provinces')
feat = provinces.first() # work on first polygon
# Landsat 8 analysis
L8 = ee.ImageCollection('LANDSAT/LC08/C01/T1_TOA') \
.filterDate(start_date, end_date) \
.filterBounds(feat.geometry()) \
L8_m = ictools.mosaicSameDay(L8)
# Now check results
L8_m = ee.ImageCollection(L8_m.toList(n_images))
L8area = valid_area(L8.first()).getInfo()['properties']['valid_area']
L8marea = valid_area(L8_m.first()).getInfo()['properties']['valid_area']
print(L8area)
shows 3,943,647,359,
while print(L8marea)
shows 3,161,128.
To reiterate: the L8 image completely covers the polygon (and the area of the polygon is roughtly 3.9x10^9, matching the L8area value), so mosaicing additional images to it should not change its area value.
I also know that the images come from the same date, so my question is: why is the area different after running the sameDayMosaic tool? My best guess is that the tool does not preserve the CRS or CRS transform, although I thought GEE would handle those things under the hood through the ee.Image.pixelArea() function.
Let me know if there's any more info you need :)
Hi,
I'm contacting you because I have a problem with geetools.
I'm working on jupyter with anaconda 3.
Here my code :
## Init ##
from IPython.display import Image
import ee
import folium
from geetools import batch
import datetime
ee.Initialize()
parcelle = ee.Geometry.Rectangle([[xxxxxxx,xxxxxx],[xxxxxxx,xxxxxxx]])
def addNDVI(image):
ndvi=(image.expression('float(b("B4") - b("B3")) / (b("B4") + b("B3"))')
.rename('NDVI'))
return image.addBands(ndvi)
collection = (ee.ImageCollection('LANDSAT/LE07/C01/T1_SR')
.filterDate(datetime.datetime(2000,1,1),
datetime.datetime(2020,3,3))
.filterBounds(parcelle)
.select(['B4', 'B3'])
.map(addNDVI).select('NDVI'));
batch.ImageCollection.toDrive(collection, 'Landsat7',
{scale: 30,
region: parcelle.getInfo()["coordinates"],
type: 'float'})
And I'm getting the following error:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-7-fde281556d5e> in <module>
----> 1 batch.ImageCollection.toDrive(collection, 'Landsat7',
2 {scale: 30,
3 region: parcelle.getInfo()["coordinates"],
4 type: 'float'})
AttributeError: module 'geetools.batch' has no attribute 'ImageCollection'
Could you help me ?
Hi there!
I am trying the notebooks attached with the master release. Just upgraded geetools using pip but I keep getting a:
ImportError: cannot import name 'composite'
I can import all other modules fine. I saw that the module composite was added 3 days ago, maybe it isn't yet available on pip?
Thanks!
Are there corresponding functions in gee_tools?
Hi!
Whenever I try to run the ESA cloud masking on a sentinel collection I get the error: "object of type 'Image' has no len()"
my code is as follows:
`imagecol = ee.ImageCollection("COPERNICUS/S2_SR").filterDate('2019-02-21', '2019-03-23').filterBounds(riv);
def s2cloud(imagecol):
return imagecol.updateMask(cloud_mask.sentinel2(imagecol))
imagecol2 = imagecol.map(s2cloud);
`
I really appreciate any help- thanks for this awesome toolbox!
Works fine in python 2
# pip install geetools
Collecting geetools
Using cached https://files.pythonhosted.org/packages/51/83/3f6f6cc185902d27468dcbef69736d12244386ec26439cabb6a09a30f471/geetools-0.1.7.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-bitz479n/geetools/setup.py", line 5, in <module>
from geetools import __version__
File "/tmp/pip-build-bitz479n/geetools/geetools/__init__.py", line 17, in <module>
from . import tools, bitreader, cloud_mask, expressions, decision_tree,\
File "/tmp/pip-build-bitz479n/geetools/geetools/tools/__init__.py", line 3, in <module>
from . import imagecollection, date, dictionary, image, number, \
File "/tmp/pip-build-bitz479n/geetools/geetools/tools/imagecollection.py", line 3, in <module>
import ee
File "/usr/local/lib/python3.6/dist-packages/ee/__init__.py", line 1, in <module>
from .main import main
File "/usr/local/lib/python3.6/dist-packages/ee/main.py", line 10, in <module>
import StringIO
ModuleNotFoundError: No module named 'StringIO'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-bitz479n/geetools/
likewise with pip2, or pip3
Hi! You are saving my life with these tools. Thank you in advance!
Sorry to ask but, I keep getting this error at the moment of exporting 10 images to my Drive. My code looks like this:
from geetools import batch
import ee
ee.Initialize()
zona_aire = ee.FeatureCollection("users/romeldrivera/zona_aire")
start = ee.Date.fromYMD(2018,1,1)
days = ee.List.sequence(0,9)
def one(d):
return start.advance(d,'day')
startDates = days.map(one)
def three(img):
return img.clip(zona_aire)
def two(m):
start = ee.Date(m)
end = ee.Date(m).advance(1,'day')
date_range = ee.DateRange(start,end)
MODISday = ee.ImageCollection('MODIS/006/MCD19A2_GRANULES').select('Optical_Depth_047').filterDate(date_range).map(three)
return(MODISday.median())
daymap = two
list_of_images = startDates.map(daymap)
mt = ee.ImageCollection(list_of_images)
tasks = batch.Export.ImageCollection.toDrive(mt, 'AOD2018', scale=30, region=zona_aire.getInfo()["coordinates"])
I run that code and get
tasks = batch.Export.ImageCollection.toDrive(mt, 'AOD2018', scale=30, region=zona_aire.getInfo()["coordinates"])
AttributeError: type object 'Export' has no attribute 'ImageCollection'
I appreciate your time and attention!
`---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
in
1 from geetools import ui
2
----> 3 Map = ui.Map()
4 Map.show()~/anaconda3/lib/python3.7/site-packages/geetools/ui/ipymap.py in init(self, tabs, **kwargs)
79
80 # Asset Manager Widget
---> 81 asset_manager = ipytools.AssetManager(self)
82
83 # Layers~/anaconda3/lib/python3.7/site-packages/geetools/ui/ipytools.py in init(self, map, **kwargs)
310 # Thumb height
311 self.thumb_height = kwargs.get('thumb_height', 300)
--> 312 self.root_path = ee.data.getAssetRoots()[0]['id']
313
314 # MapIndexError: list index out of range`
If I include the maxImgs
parameter in a call to batch.Export.imagecollection.toDrive
, I get the following error: EEException: Unknown configuration options: {'maxImgs': 20}.
.
When clicking with the inspector tool over a Feature, shows:
'ascii' codec can't encode character u'\ufffd' in position 1: ordinal not in range(128)
I am trying to export a collection of Landsat 8 images as assets but keep getting a geometry error. I originally was performing a number of operations to the scenes once selected from the landsat collection and thought maybe the geometry was getting altered in the process. In the code below, I tried to simplify the process so that the final image collection represents the least cloudy Landsat 8 scene over a selection of watersheds in Montana, during the 2018growing season, but the error still occurs.
Thanks in advance for the help and I appreciate the tools you have put together!
var huc8 = ee.FeatureCollection("USGS/WBD/2017/HUC08")
var mt = ee.FeatureCollection("TIGER/2018/States").filter(ee.Filter.equals('NAME', 'Montana'));
// Map.addLayer(mt)
var sheds = huc8.filterBounds(mt).filter(ee.Filter.or(ee.Filter.equals('name','Peoples'), ee.Filter.eq('name','Belt'), ee.Filter.eq('name','Arrow')));
// var sheds = ee.FeatureCollection('users/ryanpatrickrock/bpHuc_testData')
print(sheds);
var testFun = function(feature){
var ROI = feature.geometry();
var addCloud = function(image) {
var cloud = ee.Algorithms.Landsat.simpleCloudScore(image).select('cloud');
var cloudiness = cloud.reduceRegion({ // calculate average cloud score across the ROI i.e the watershed specified
reducer: 'mean',
geometry: ROI,
scale: 30,
maxPixels: 1e12
});
return image.set(cloudiness); // add this cloud score to the metadata of the image
};
var l8 = ee.ImageCollection("LANDSAT/LC08/C01/T1_TOA")
.filterDate('2018-05-15', '2018-09-15')
.filterBounds(ROI)
.map(addCloud)
.sort('cloud')
var scene = l8.first()
.select(['B1','B2','B3','B4','B5','B6','B7'])
.clip(ROI)
return scene
};
var sceneCol = ee.ImageCollection(sheds.map(testFun));
print(sceneCol)
Map.addLayer(sceneCol, {bands: ['B4', 'B3', 'B2'], min: 0, max: 0.5})
Map.addLayer(sceneCol.first().geometry().bounds())
print(sceneCol.first().geometry().bounds())
var batch = require('users/fitoprincipe/geetools:batch');
batch.Download.ImageCollection.toAsset(sceneCol, 'users/ryanpatrickrock', {
name: 'test_batchCollection',
scale: 30,
region: null,
maxPixels: 1e12
}
);
Hello,
I am new user of GEE, I am trying to perform a supervised classification, so when I import my shapefile, I get the following error message : "Error: Unable to transform edge (549555.928576868,3474668.316669156 to 549555.9286273334,3474668.3166489694) from Planar EPSG:4326 to Geodesic EPSG:4326 (within 4.488328883882435E-5 units)"
I didn't understand, how can I transform it from Planar EPSG:4326 to Geodesic EPSG:4326 ?
Any help, Please?
AyouCha
I tried running hollsteinMask but it requires B10 in sentinel2. It seems this was removed from the GEE collection.
Hello!
I received a similar error message as reported on fitoprincipe/geetools-code-editor#2 when trying to batch download an image collection, except on python.
Here's the code:
`import ee
ee.Initialize()
from geetools import batch
feature = ee.Geometry.Polygon([[-20.3014885, -51.5035369], [-20.84233950, -51.6313196], [-20.6922002, -51.2336120], [-20.5804476, -52.1965132]]), {'site': 1}
fc = ee.FeatureCollection(feature)
collection = ee.ImageCollection("COPERNICUS/S5P/NRTI/L3_CLOUD").select('cloud_optical_depth').filterDate('2019-09-10', '2019-09-13').filterBounds(fc.geometry())
image = collection.mosaic()
tasklist = batch.image.toDriveByFeature(image, collection=fc, property='site', folder='Result_test_S5P', name='test', scale=305, dataType='float')
tasklist.start()`
And the Error:
`---------------------------------------------------------------------------
EEException Traceback (most recent call last)
in
6
7 feature = ee.Geometry.Polygon([[-20.3014885, -51.5035369], [-20.84233950, -51.6313196], [-20.6922002, -51.2336120], [-20.5804476, -52.1965132]]), {'site': 1}
----> 8 fc = ee.FeatureCollection(feature)
9 collection = ee.ImageCollection("COPERNICUS/S5P/NRTI/L3_CLOUD").select('cloud_optical_depth').filterDate('2019-09-10', '2019-09-13').filterBounds(fc.geometry())
10 image = collection.mosaic()
~\AppData\Local\Continuum\miniconda3\envs\S5P-37\lib\site-packages\ee\computedobject.py in call(cls, *args, **kwargs)
30 return args[0]
31 else:
---> 32 return type.call(cls, *args, **kwargs)
33
34
~\AppData\Local\Continuum\miniconda3\envs\S5P-37\lib\site-packages\ee\featurecollection.py in init(self, args, opt_column)
62 super(FeatureCollection, self).init(
63 apifunction.ApiFunction.lookup('Collection'), {
---> 64 'features': [feature.Feature(i) for i in args]
65 })
66 elif isinstance(args, ee_list.List):
~\AppData\Local\Continuum\miniconda3\envs\S5P-37\lib\site-packages\ee\featurecollection.py in (.0)
62 super(FeatureCollection, self).init(
63 apifunction.ApiFunction.lookup('Collection'), {
---> 64 'features': [feature.Feature(i) for i in args]
65 })
66 elif isinstance(args, ee_list.List):
~\AppData\Local\Continuum\miniconda3\envs\S5P-37\lib\site-packages\ee\computedobject.py in call(cls, *args, **kwargs)
30 return args[0]
31 else:
---> 32 return type.call(cls, *args, **kwargs)
33
34
~\AppData\Local\Continuum\miniconda3\envs\S5P-37\lib\site-packages\ee\feature.py in init(self, geom, opt_properties)
75 # turning out to be GeoJSON.
76 super(Feature, self).init(feature_constructor, {
---> 77 'geometry': geometry.Geometry(geom),
78 'metadata': opt_properties or None
79 })
~\AppData\Local\Continuum\miniconda3\envs\S5P-37\lib\site-packages\ee\computedobject.py in call(cls, *args, **kwargs)
30 return args[0]
31 else:
---> 32 return type.call(cls, *args, **kwargs)
33
34
~\AppData\Local\Continuum\miniconda3\envs\S5P-37\lib\site-packages\ee\geometry.py in init(self, geo_json, opt_proj, opt_geodesic, opt_evenOdd)
81
82 if not Geometry._isValidGeometry(geo_json):
---> 83 raise ee_exception.EEException('Invalid GeoJSON geometry.')
84
85 super(Geometry, self).init(None, None)
EEException: Invalid GeoJSON geometry.`
I also tried importing the geometry as shapefile and as a geojson through geopandas, still got the same error. Not sure if this is a gee-tools error or an ee error. Can you help me?
I am new to python so hopefully it's just me doing something wrong...
Thanks in advance.
Hi Rodrigo,
I am also trying to use geetools to download imageCollection. However, I cannot find col2drive function in geetools.tools. My code is below. Thank you for your help and this powerful tool.
from geetools import tools import ee ee.Initialize() S1_images=ee.ImageCollection('COPERNICUS/S1_GRD').filterDate('2017-08-27','2017-08-29') task = tools.col2drive(S1_images,r'D:\result')
I got the error message
AttributeError: module 'geetools.tools' has no attribute 'col2drive'
I am trying to install the Geetools library on datalab in docker, and the error occurs:
ImportError: No module named shapefile.
Someone has had something similar.
thank you!!
Hello,
I've tried exporting a couple of different ways but seem to have no luck. I tried using the exportByFeat.ipynb as a guide and followed the script as is but I get the same result. The correct folder is created in my drive but there is nothing inside. Am I doing anything wrong?
I am a user of gee_tools. I usually export image collection to google drive. However, when the number of the task in the queue is large than 3000, the program will be halted. I wonder if there is any way to check the number of the task in the queue and pause the program when the number is large?
Thanks for your kind help and looking forward to your reply.
Best,
Enze
Hi
great tools , thanks a lot! Is it possible that addLayer
cannot show Features
on map, only geometries? I got warning message TypeError: addGeoJson() takes exactly 2 arguments (1 given) on a Feature. When using on FeatureCollection, I get: addLayer doesn't support adding the specified object to the map.
See:
import ee
ee.Initialize()
rectangle = ee.Geometry.Rectangle([-40, -20, 40, 20]);
rectangle_f = ee.Feature(rectangle)
rectangle_fc = ee.FeatureCollection(rectangle_f)
from geetools import ipymap
Map2 = ipymap.Map()
Map2.show()
Map2.addLayer(rectangle, {}, 'rectangle')
Map2.centerObject(rectangle)
All works fine. But now these two will throw an error?
Map2.addLayer(rectangle_f, {}, 'rectangle feature')
Map2.addLayer(rectangle_fc, {}, 'rectangle feature')
Hi there -
Sorry to bug you with, what I am guessing, is a very basic question! I've been trying to piece together your code snippets from various StackExchange posts, but am likely missing something obvious. I'm using Spyder and have imported your gee_tools.
My task is simple! I want to download multiple L8 images (as many as available) of a small region in Greenland but am having trouble with the iterating part. I think I "should" be able to call your gee.ExportCol command as well, but I'm not sure where that fits in.
kgl = ee.Geometry.Polygon(
[[[-33.18145751953125, 68.63655326841952],
[-32.79144287109375, 68.52722953172731],
[-32.65960693359375, 68.58245935169703],
[-32.95623779296875, 68.68951879149195]]])
start = '2016-05-01'
stop = '2016-06-01'
images = ee.ImageCollection("LANDSAT/LC8_L1T_TOA")
.filterBounds(kgl)
images = images.filterDate(start, stop)
listOfImages = images.toList(images.size());
After this snippet, I tried lots of variations of the task= ee.batch.Export.image.toDrive and gee.ExportCol...but I don't think I have my list set up correctly. Would appreciate any suggestions!
Hi Rodrigo,
When i try your code on google colab,an error occured.
IndexError Traceback (most recent call last)
in ()
3 i = ee.Image('COPERNICUS/S2/20181122T142749_20181122T143353_T18GYT').clip(aoi)
4 grid = tools.image.toGrid(i, 3, geometry=aoi)
----> 5 Map = Map()
6
7 Map.show()
1 frames
/usr/local/lib/python3.6/dist-packages/ipygee/assets.py in init(self, map, **kwargs)
20 # Thumb height
21 self.thumb_height = kwargs.get('thumb_height', 300)
---> 22 self.root_path = ee.data.getAssetRoots()[0]['id']
23
24 # Map
IndexError: list index out of range
Could you please help me about it?
Thank you for making this functionality available.
I'm been trying to export some images on my google cloud storage, but am running into trouble.
Below is an example of my run when I try to upload 9 images
BUCKET = "detect_proto"
imageExportFormatOptions = {
'patchDimensions': [75, 75],
'maxFileSize': 100000000,
'compressed': True
}
batch.imagecollection.toCloudStorage(newImgCol, bucket=BUCKET, folder="geetools", scale=10,
fileFormat = 'TFRecord', verbose=True,
formatOptions = imageExportFormatOptions)
This will yield the following output
adding task 0 to list
adding task 1 to list
adding task 2 to list
adding task 3 to list
adding task 4 to list
adding task 5 to list
adding task 6 to list
adding task 7 to list
adding task 8 to list
But when I check my cloud storage, only 1 file has been uploaded
gsutil ls gs://detect_proto/geetools
gs://detect_proto/geetools/0.json
gs://detect_proto/geetools/0.tfrecord.gz
I'm a bit confused why only 1 file has been uploaded, any thoughts on how to investigate this would be greatly appreciated.
CC @MarcCoru @fitoprincipe
Hi,
I've been using this code for a long time, and today it stopped working https://code.earthengine.google.com/2e38ff1b7dc0d006715a085cb469ec90 . If shortly:
Today, batch download stopped working with step 3: I get notification "An internal error has occurred (request: 44215952-a82b-4627-9be7-844555231e66)"; at the same time, the batch download runs only for 2-5 files (out of 12).
If I skip step 3, the code runs properly and the batch download works fine. I have reported this problem in the GEE feedback field, but I don't know how long it will take. Maybe you could help me understand the reason for the problem and provide some solution?
I really appreciate any help you can provide.
Hello @fitoprincipe , Would you be so kind as to help me?
I am interested in downloading Sentinel-1 images using this script:
import ee
ee.Authenticate()
ee.Initialize()
import geetools
region = ee.Geometry.Rectangle([-64.1569659949999931,-8.8276458170000005,
-63.7711738149999974,-8.5146591039999997])
start_date = ee.Date('2019-05-01')
finish_date = ee.Date('2019-05-7')
orbit = 'ASCENDING'
collectionS1 = ee.ImageCollection('COPERNICUS/S1_GRD')\
.filter(ee.Filter.listContains('transmitterReceiverPolarisation', 'VV'))\
.filter(ee.Filter.eq('instrumentMode', 'IW'))\
.filterDate(start_date, finish_date)\
.filterBounds(region)
collectionVV = collectionS1.select('VV')
folder = 'GEE'
geetools.batch.imagecollection.toDrive(collectionVV, folder=folder, scale=10, maxPixels=1e13,
region=region)
It works well, however two folders with the same name are downloaded, and the total of the downloaded images are in both folders.
Why does this happen?
Thank you so much for your time
Best,
Yonatan
Originally posted by @ytarazona in #44 (comment)
hey @fitoprincipe I'm working with geetools in jupyter, but I've got some problems with visualising Copernicus Sentinel 2 imagery. It works well with Landsat. I also noticed I cannot run ee.mapclient, it gives me this error:
Exception in thread Thread-4: Traceback (most recent call last): File "/home/maykef/anaconda3/lib/python3.7/threading.py", line 917, in _bootstrap_inner self.run() File "/home/maykef/anaconda3/lib/python3.7/site-packages/ee/mapclient.py", line 125, in run self.parent = Tkinter.Tk() File "/home/maykef/anaconda3/lib/python3.7/tkinter/__init__.py", line 2023, in __init__ self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use) _tkinter.TclError: no display name and no $DISPLAY environment variable
keep the mask data until the end of the process and check if the resulting value is 0 and it was a masked value, mask it again, so not to keep data with 0 value
cloud masking leads into this error
"
HttpError: <HttpError 400 when requesting https://earthengine.googleapis.com/v1alpha/projects/earthengine-legacy/value:compute?prettyPrint=false&alt=json returned "Image.select: Pattern 'pixel_qa' did not match any bands.". Details: "Image.select: Pattern 'pixel_qa' did not match any bands.">"
is it due to Landsat SR band name convention changed as 'QA_PIXEL',
I followed the code in your notebook.
col = ee.ImageCollection('COPERNICUS/S2')
site = ee.Geometry.Point([-72,-42])
col = col.filterBounds(site).filterMetadata('CLOUD_COVERAGE_ASSESSMENT','less_than',40)
image =ee.Image(col.first())
geom = image.geometry()
visParam = {'bands':['B8','B11','B4'], 'min':0,'max':5000}
# addLayer has a parameter called inspect
inspect = {'data':image,'reducer':'mean'}
from geetools import maptool
Map = maptool.Map()
Map.addLayer(image,visParam,'Sentinel 2')
Map.centerObject(image)
Map.addLayer(geom, name = 'Image Boundary',inspect = inspect)
Map.show()
AttributeError: 'dict' object has no attribute 'iteritems'
from geetools import ipymap
Map2 = ipymap.Map()
Map2.show()
C:\ProgramData\Anaconda3\lib\site-packages\geetools\ipymap.py in ()
18 get_image_tile, get_geojson_tile, get_bounds, get_zoom,
19 create_html, feature_properties_output
---> 20 import maptool
21 from . import ipytools
22 import threadingModuleNotFoundError: No module named 'maptool'
This parameter has to be modified to match the new structure
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.