spatialstatisticsupna / rgistools Goto Github PK

View Code? Open in Web Editor NEW

51.0 6.0 15.0 4.61 MB

Tools for Downloading, Customizing, and Processing Time Series of Satellite Images from Landsat, MODIS, and Sentinel

License: GNU General Public License v3.0

R 98.94% TeX 1.06%

satellite-imagery satellite-data gap-filling smoothing

rgistools's People

Contributors

Stargazers

Watchers

Forkers

rsbivand sonthuybacha xolomo mmontesinosanmartin kant earthcli ritaaframos yangxhcaf drroad mattols sumesh1 hydroinfo-gis m89j sharadgupta27 aylinsahin

rgistools's Issues

Please remove dependencies on rgdal, rgeos, and/or maptools

This package depends on (depends, imports or suggests) raster and one or more of the retiring packages rgdal, rgeos or maptools (https://r-spatial.org/r/2022/04/12/evolution.html, https://r-spatial.org/r/2022/12/14/evolution2.html). Since raster 3.6.3, all use of external FOSS library functionality has been transferred to terra, making the retiring packages very likely redundant. It would help greatly if you could remove dependencies on the retiring packages as soon as possible.

I think that rgdal is suggested only to satisfy the former needs of raster now met by terra.

upcoming stars release breaks RGISTools, through tmap

@mtennekes
I submnitted stars 0.5-0 to CRAN, which causes a reverse depency error in RGISTools (see here):

Changes to worse in reverse depends:

Package: RGISTools
Check: examples
New result: ERROR
  Running examples in â€˜RGISTools-Ex.Râ€™ failed
  The error most likely occurred in:
  
  > base::assign(".ptime", proc.time(), pos = "CheckExEnv")
  > ### Name: genCompositions
  > ### Title: Create image compositions from a time series of satellite images
  > ### Aliases: genCompositions
  > 
  > ### ** Examples
  > 
  > # loading NDVI images of Navarre
  > data("ex.ndvi.navarre")
  > # Ploting the images: clouds are found
  > genPlotGIS(ex.ndvi.navarre)
  Warning: col specification in tm_raster is ignored, since stars object contains a 3rd dimension, where its values are used to create facets
  Error in dim.stars(x) : length(d) == length(dim(x[[1]])) is not TRUE
  Calls: <Anonymous> ... do.call -> plot_1 -> nrow -> dim -> dim.stars -> stopifnot
  Execution halted

The cause for this is in tmap/tmaptools, and if I update tmap and tmaptools to their github versions this error goes away. So I can see two options for this:

change RGISTools (e.g. out-comment all calls to genPlotGIS from the examples, and add a warning when using it that tmap/tmaptools should have a minimum version and be installed from github), or
update tmap and/or tmaptools on CRAN

Martijn, when do you have in mind you could submit new tmap/tmaptools, is this waiting for a CRAN release of stars?

earthexplorer Retired Api

endpoint changed from https://earthexplorer.usgs.gov/inventory/json/v/1.4.0/ to https://m2m.cr.usgs.gov/api/api/json/stable/

renamed error parameter to errorCode in login method response:
{
"requestId": xxx,
"version": "stable",
"data": "xxx",
"errorCode": null,
"errorMessage": null,
"sessionId": xxx
}

Pricing

Hello,
I'm a beginner on remote sensing but i'm trying to obtain some NDVI maps of places. does anyone have informations regarding the pricing the download the satellite images from sentinel ?
Thank you

Cannot download Landsat8 images through RGISTools R package

I've been having a hard time trying to automate some Landsat 8 image download and processing for my project.

At first, I tried to use getSpatialData package to do what I need from R, but it seems the authors aren't maintaining and supporting the library recently.

Now, I'm trying to acomplish my goals by using RGISTools package, but I'm reading the article the authors wrote about the package and testing in real time here in Rstudio and when I try to run some functions like lsDownload(), and it keeps returning an error:

Downloading the images in: C:/PROJETOS_PRINCIPAIS2/AUTOMATE_SATELITE_IMAGE/RGISTools/Teste/landsat8/downloads/Landsat8/raw

Starting Landsat level 1 download process...

Error in charToRaw(enc2utf8(val)) :

  argument must be a character vector of length 1

Here is my code for you to take a look if I'm missing something:

##### Import libraries #####

library(raster)

library(sp)

library(sf)

library(RGISTools)

##### Set the working directory #####

scriptWD <- 'C:/PROJETOS_PRINCIPAIS2/AUTOMATE_SATELITE_IMAGE/RGISTools/Teste'

setwd(scriptWD)

##### Import area of interest #####

hortoMG <- st_read('mogiGuacu.shp')

##### Searching for images #####

search <- lsSearch(AppRoot = scriptWD,

                    product = 'LANDSAT_8_C1',

                    dates = as.Date('2020-07-01') + seq(0, 15, 1),

                    region = hortoMG,

                    cloudCover = c(0, 80),

                    username = '*****',

                    password = '*****')


##### Search Preview #####

lsPreview(searchres = search)

##### Create the wd for landsat images downloads #####

wdLandsat <- file.path(scriptWD, 'landsat8')

wdLandsatDownload <- file.path(wdLandsat, 'downloads')

##### Baixando imagens #####

testDownload <- lsDownload(searchres = search,

                    AppRoot = wdLandsatDownload,

                    lvl = 1,

                    product = 'LANDSAT_8_C1',

                    extract.tif = TRUE,

                    username = '*****',

                    password = '*****',

                    nattempts = 5)

CRAN?

Is there a plan to put this back on CRAN? We would like to include it in the Agriculture CRAN task view.

Thank you,

Julia

Error in .tiff conversion function in modDownSearch

I am getting an error when I try to download the "MOD09GA" band for uttrakhand. .hdf to .tiff conversion function is not working properly

Suggest to rename argument in senSearch

I noticed when trying to retrieve URLs from senSearch that the argument name "lonlat" and its description is misleading. I was searching for the point 97.65317 (lon), 24.15859 (lat), so in the "lonlat" argument I put it in this order. However, the function returned NULL saying there were no images, even though a manual search in Copernicus SciHub showed there was.

I tried switching the argument to lat, then lon, and the function worked.

Can I please suggest you rename the argument to be "latlon" and update the description?

Here is the code. First one does not work.

senSearch(
            username="****",
            password="*****", 
            verbose=TRUE, 
            platform="Sentinel-2", 
            product="S2MSI1C", 
            lonlat=c(97.65317, 24.15859), # c(lon, lat)
            startDate=as.Date("2015-01-01"), 
            endDate=as.Date("2018-12-17")
)

Adapting dates.
Adding platform name.
[1] "Adding query intersects"
Adding query intersects
Added product type.
https://scihub.copernicus.eu/dhus/search?q=beginposition:[2015-01-01T00:00:00.000Z%20TO%202018-12-17T23:59:59.999Z]%20AND%20platformname:Sentinel-2%20AND%20footprint:%22intersects(97.65317,%2024.15859)%22%20AND%20producttype:S2MSI1C&format=json&rows=100
Search Total result: 
There is no images in the response, check spatial and temporal arguments.
NULL

This one works.

senSearch(
            username="****",
            password="*****", 
            verbose=TRUE, 
            platform="Sentinel-2", 
            product="S2MSI1C", 
            lonlat=c(24.15859, 97.65317), # c(lat, lon)
            startDate=as.Date("2015-01-01"), 
            endDate=as.Date("2018-12-17")
)

Adapting dates.
Adding platform name.
[1] "Adding query intersects"
Adding query intersects
Added product type.
https://scihub.copernicus.eu/dhus/search?q=beginposition:[2015-01-01T00:00:00.000Z%20TO%202018-12-17T23:59:59.999Z]%20AND%20platformname:Sentinel-2%20AND%20footprint:%22intersects(24.15859,%2097.65317)%22%20AND%20producttype:S2MSI1C&format=json&rows=100
Search Total result: 121
Image result 1 Name:S2A_MSIL1C_20150815T041056_N0204_R047_T47QLG_20150815T041524
Image result 1 Url:https://scihub.copernicus.eu/dhus/odata/v1/Products('b3fc9f4e-f857-4f65-abcd-af74551b7c50')/$value
...
...

Is it possible to update (or translate) genSmoothingIMA function for SpatRaster object of terra?

Dear All,
Recently, I downloaded a large LST data from AppEEARS with NetCDF4 format (projection “PROJ.4:+proj=longlat +datum=WGS84 +no_defs “ is selected) for gap filling. When the large NetCDF4 file is loaded by raster::brick in R, An error occurred as “Error in CRS(x) : NA”.
The NetCDF4 file can be loaded by terra::rast without error. However, genSmoothingIMA function requires RasterBrick object.
So, is it possible to update (or translate) genSmoothingIMA function for SpatRaster object of terra?

Cryptic error in lsSearch

I'm trying to download Landsat bands for Indonesia, but I keep getting an obscure (not helpful) error message. An example below. Note that in the real example I'm of course using my own EarthExplorer/USGS username and password, as character strings. I'm currently logged into EarthExplorer with those credentials.

#Import a basic map of the world to try to pull down data associated with Indonesia.
require(maptools)
require(sf)
#Pull in a shapefile of simplified country outlines
data(wrld_simpl)
class(wrld_simpl)

#Make this a "sf" class object
wrld_simpl=st_as_sf(wrld_simpl)
class(wrld_simpl)

#Shapefile of just Indonesia.
require(dplyr)
indonesia = wrld_simpl %>% filter(NAME=="Indonesia")

#Plot the shapefile. It's a real place.
require(ggplot)
ggplot(indonesia) + geom_sf()

#############################
#Attempt to search for Landsat 8 data over Indonesia, specifying the area of interest 2 different ways:

#From the documentation of "lsSearch"
# region: a ‘Spatial*’, projected ‘raster*’, o r ‘sf’ class object
#       defining the area of interest. This argument is mandatory if
#       ‘pathrow’, ‘extent’, or ‘lonlat’ are not defined.
class(indonesia)

#Literally the example given in the help file, just using my username and password and the Indonesia shapefile.
sres <- lsSearch(product = "LANDSAT_8_C1",
                      startDate = as.Date("01-01-2011", "%d-%m-%Y"),
                      endDate = as.Date("31-12-2013", "%d-%m-%Y"),
                      username = earthexplorer_username,
                      password = earthexplorer_password,
                      region = indonesia)
#Returns a cryptic error:
#Error in if (res$error != "") { : argument is of length zero

#From the documentation of "lsSearch"
# extent: an ‘extent’, ‘Raster*’, or ‘Spatial*’ object
#               representing the region of interest with
#               longitude/latitude (EPSG:4326) coordinates. This argument
#               is mandatory if ‘region’, ‘pathrow’ or ‘lonlat’ are not
#               defined.
require(raster)
class(extent(wrld_simpl))

#Literally the example given in the help file, just using my username and password and the Indonesia shapefile.
sres <- lsSearch(product = "LANDSAT_8_C1",
                      startDate = as.Date("01-01-2011", "%d-%m-%Y"),
                      endDate = as.Date("31-12-2013", "%d-%m-%Y"),
                      username = earthexplorer_username,
                      password = earthexplorer_password,
                      extent = extent(wrld_simpl))
#Exact same cryptic error.

I'm not sure what the problem is, but whatever it is, returning an error that tells me something other than res$error != "" would be really helpful. Given the naming, I suspect there was an error downloading the files? If so, returning an error message like "Unable to download files, please check credentials" or "EarthData server not responding" would be really helpful!

Error running the lsDownload function

I am using the lsSearch and lsDownload function for some weeks now to obtain the landsat images from several seasons and years. It was working fine (mostly) but now I get an error I can't figure out:

#search images with cloud cover less than 30%
PB2014 <- lsSearch(product = "LANDSAT_8_C1",
startDate = as.Date("15-07-2014", "%d-%m-%Y"),
endDate = as.Date("15-09-2014", "%d-%m-%Y"),
cloudCover = c(0,30),
username = "username",
password = "password",
extent = postbreeding) #area, may be raster or poligon

#download the data
lsDownload(searchres = PB2014, #with the test layer
username = "username",
password = "password",
untar = FALSE,
raw.rm = FALSE,
AppRoot = wdir_pb14,
nattempts = 5)

Downloading the images in: /home/ritaramos/Desktop/NDVI/postbreebing2014/Landsat8/raw
Starting Landsat level 1 download process...
Error in charToRaw(enc2utf8(val)) :
argument must be a character vector of length 1

was there a change in the package recently that requires a change in the script?

All the best,

senDownload ERROR:Error in curl_download(image.url, destfile = downPath, handle = c.handle): HTTP error 403.

Hello,

I am trying to download Sentinel-2 data with your package. I've tried a few different options, but nothing is allowing me to get the data downloaded.

wdir <- file.path(getwd(),"Sentinel-2")

sres <- senDownload(searchres = CHsent_16,
                    username = "MY_USER_NAME",
                    password = "MY_PASSWORD",
                    AppRoot = wdir,
                    unzip = TRUE)

It looks like the files start to download into the raw - I see a file appear and then it disappears. I get the following:

Downloading the images in: C:/Users/ecolo/AppData/Local/Temp/Rtmpqo3IpQ/Sentinel-2//raw
Downloading image S2A_MSIL1C_20160629T162332_N0204_R040_T19WFM_20160629T162335 (1/12)
https://scihub.copernicus.eu/dhus/odata/v1/Products('00c5d64c-8f38-4739-9616-ecab58de5956')/Checksum/Value/$value
Get md5: 1867DD4FEA635B700C54F5CD4D97043B
Error cheking S2A_MSIL1C_20160629T162332_N0204_R040_T19WFM_20160629T162335 file md5: 1867DD4FEA635B700C54F5CD4D97043B
Downloading the images in: C:/Users/ecolo/AppData/Local/Temp/Rtmpqo3IpQ/Sentinel-2//raw
Downloading image S2A_MSIL1C_20160629T162332_N0204_R040_T19WFM_20160629T162335 (1/1)
https://scihub.copernicus.eu/dhus/odata/v1/Products('00c5d64c-8f38-4739-9616-ecab58de5956')/Checksum/Value/$value
Get md5: 1867DD4FEA635B700C54F5CD4D97043B
Error cheking S2A_MSIL1C_20160629T162332_N0204_R040_T19WFM_20160629T162335 file md5: 1867DD4FEA635B700C54F5CD4D97043B

etc...

Downloading the images in: C:/Users/ecolo/AppData/Local/Temp/Rtmpqo3IpQ/Sentinel-2//raw
Downloading image S2A_MSIL1C_20160629T162332_N0204_R040_T19WFM_20160629T162335 (1/1)
https://scihub.copernicus.eu/dhus/odata/v1/Products('00c5d64c-8f38-4739-9616-ecab58de5956')/Checksum/Value/$value
Get md5: 1867DD4FEA635B700C54F5CD4D97043B
Error cheking S2A_MSIL1C_20160629T162332_N0204_R040_T19WFM_20160629T162335 file md5: 1867DD4FEA635B700C54F5CD4D97043B
Error downloading S2A_MSIL1C_20160629T162332_N0204_R040_T19WFM_20160629T162335
The images have been unzipped in: C:/Users/ecolo/AppData/Local/Temp/Rtmpqo3IpQ/Sentinel-2//unzip
The images have been unzipped in: C:/Users/ecolo/AppData/Local/Temp/Rtmpqo3IpQ/Sentinel-2//unzip
The images have been unzipped in: C:/Users/ecolo/AppData/Local/Temp/Rtmpqo3IpQ/Sentinel-2//unzip
The images have been unzipped in: C:/Users/ecolo/AppData/Local/Temp/Rtmpqo3IpQ/Sentinel-2//unzip
Downloading image S2A_MSIL1C_20160630T155412_N0204_R054_T19WFM_20160630T155407 (2/12)
ERROR:Error in curl_download(image.url, destfile = downPath, handle = c.handle): HTTP error 403.

Downloading image S2A_MSIL1C_20160706T161352_N0204_R140_T19WFM_20160706T161347 (3/12)
ERROR:Error in curl_download(image.url, destfile = downPath, handle = c.handle): HTTP error 403.

What might be the issue here?

Thank you,

Mark Thompson

modPreview fails due to lack of CRS, even though it seems to be there

When I run this code:

library(RGISTools)
library(rgdal)

bbox_string <- '
{
"type": "FeatureCollection",
"name": "KingCounty",
"crs": { "type": "name", "properties": { "name": "urn:ogc:def:crs:OGC:1.3:CRS84" } },
"features": [
{ "type": "Feature", "properties": { "MINX": -122.54204074767026, "MINY": 47.084628061013817, "MAXX": -121.06594568541907, "MAXY": 47.780581394066793, "CNTX": -121.80399321654465, "CNTY": 47.432604727540308, "AREA": 1.0272932784767566, "PERIM": 4.344096790608333, "HEIGHT": 0.69595333305297657, "WIDTH": 1.4760950622511899 }, "geometry": { "type": "Polygon", "coordinates": [ [ [ -122.542040747670256, 47.084628061013817 ], [ -121.065945685419067, 47.084628061013817 ], [ -121.065945685419067, 47.780581394066793 ], [ -122.542040747670256, 47.780581394066793 ], [ -122.542040747670256, 47.084628061013817 ] ] ] } }
]
}
'

bbox <- readOGR(bbox_string)
sres <- modSearch(product = "MOD09GA",
                  dates = seq(as.Date("2016-01-01"),as.Date("2016-01-31"),by=1),
                  region = bbox)

modPreview(searchres = sres, dates = as.Date("2016-01-01"))

I get an error:

Warning message in CPL_crs_from_input(x):
“GDAL Error 1: PROJ: proj_create_from_database: crs not found”
Error in CPL_transform(x, crs, aoi, pipeline, reverse): crs not found: is it missing?
Traceback:

1. modPreview(searchres = sres, dates = as.Date("2016-01-01"))
2. .modPreviewRecursive(searchres = searchres, n = 1, lpos = lpos, 
 .     add.Layer = add.Layer, verbose = verbose, ...)
3. extent(st_transform(mod.tiles[mod.tiles$Name == paste0("h:", 
 .     ho, " v:", ve), ], crs = st_crs(54008)))
4. st_transform(mod.tiles[mod.tiles$Name == paste0("h:", ho, " v:", 
 .     ve), ], crs = st_crs(54008))
5. st_transform.sf(mod.tiles[mod.tiles$Name == paste0("h:", ho, 
 .     " v:", ve), ], crs = st_crs(54008))
6. st_transform(st_geometry(x), crs, ...)
7. st_transform.sfc(st_geometry(x), crs, ...)
8. structure(CPL_transform(x, crs, aoi, pipeline, reverse), single_type = NULL, 
 .     crs = crs)
9. CPL_transform(x, crs, aoi, pipeline, reverse)

Here is the ROI as a SpatialPolygonsDataFrame, with a fairly standard CRS proj4 string.

class       : SpatialPolygonsDataFrame 
features    : 1 
extent      : -122.542, -121.0659, 47.08463, 47.78058  (xmin, xmax, ymin, ymax)
crs         : +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0 
variables   : 10
names       :             MINX,             MINY,              MAXX,             MAXY,              CNTX,             CNTY,             AREA,            PERIM,            HEIGHT,            WIDTH 
value       : -122.54204074767, 47.0846280610138, -121.065945685419, 47.7805813940668, -121.803993216545, 47.4326047275403, 1.02729327847676, 4.34409679060833, 0.695953333052977, 1.47609506225119

senDownSearch

Hi,

I am trying to run this code snippet with using senDownSearch function from RGIStools, but it throws back an error. I am just following the tutorial steps. The username and password are working fine, they was tested previously.
Any help would be appreciated. Thanks in advance, JL

senDownSearch(startDate = as.Date("2018210", "%Y%j"), endDate = as.Date("2018218", "%Y%j"), platform = "Sentinel-2", extent = ex.navarre, product = "S2MSI1C", pathrow = c("R094"), username = "xxxxx", password = "*****", AppRoot = wdir)

6 tiles found! Starting the download process...
Error in senDownload(searchres = senURL, username = username, password = password, :
A response from sentinel search function is needed.