Code Monkey home page Code Monkey logo

hddtools's Introduction

hddtools: Hydrological Data Discovery Tools

DOI status

CRAN Status Badge CRAN Total Downloads CRAN Monthly Downloads

R-CMD-check codecov.io

hddtools stands for Hydrological Data Discovery Tools. This R package is an open source project designed to facilitate access to a variety of online open data sources relevant for hydrologists and, in general, environmental scientists and practitioners.

This typically implies the download of a metadata catalogue, selection of information needed, a formal request for dataset(s), de-compression, conversion, manual filtering and parsing. All those operations are made more efficient by re-usable functions.

Depending on the data license, functions can provide offline and/or online modes. When redistribution is allowed, for instance, a copy of the dataset is cached within the package and updated twice a year. This is the fastest option and also allows offline use of package’s functions. When re-distribution is not allowed, only online mode is provided.

Installation

Get the stable version from CRAN:

install.packages("hddtools")

Or the development version from GitHub using the package remotes:

install.packages("remotes")
remotes::install_github("ropensci/hddtools")

Load the hddtools package:

library("hddtools")

Data sources and Functions

The package contains functions to interact with the data providers listed below. For examples of the various functionalities see the vignette.

  • KGClimateClass: The Koppen Climate Classification map is used for classifying the world’s climates based on the annual and monthly averages of temperature and precipitation.

  • GRDC: The Global Runoff Data Centre (GRDC) provides datasets for all the major rivers in the world.

  • Data60UK: The Data60UK initiative collated datasets of areal precipitation and streamflow discharge across 61 gauging sites in England and Wales (UK).

  • MOPEX: This dataset contains historical hydrometeorological data and river basin characteristics for hundreds of river basins in the US.

  • SEPA: The Scottish Environment Protection Agency (SEPA) provides river level data for hundreds of gauging stations in the UK.

Meta

  • This package and functions herein are part of an experimental open-source project. They are provided as is, without any guarantee.
  • Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.
  • Please report any issues or bugs.
  • License: GPL-3
  • This package was reviewed by Erin Le Dell and Michael Sumner for submission to ROpenSci (see review here) and the Journal of Open Source Software (see review status here).
  • Cite hddtools: citation(package = "rdefra")

ropensci_footer

hddtools's People

Contributors

arfon avatar cvitolo avatar dhugpeter avatar jeroen avatar karthik avatar noamross avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

hddtools's Issues

use releases for each CRAN version

Hi @cvitolo - i noticed you have used releases in the past - but the most recent CRAN version doesn't have a v0.6.0 git tag or release notes for that tag - can you add your news bits for that version to a release?

KGClimate-class problem

I see this problem, happy to help figure it out but for now I'll just post the details.

library(hddtools)
# Define a bounding box
bbox <- list(lonMin=-10,latMin=48,lonMax=5,latMax=62)

# Define a temporal extent
timeExtent <- seq(as.Date('2012-01-01'), as.Date('2012-03-31'), by='months')

KGClimateClass(bbox,updatedBy='Peel')
# Error in readBin(tarfile, "raw", n = 3L) : 
#   can only read from a binary connection
# In addition: Warning message:
#   In file(con, "rb") :
#   file("") only supports open = "w+" and open = "w+b": using the former

devtools::session_info()
# Session info -----------------------------------------------------------------------
#   setting  value                                      
# version  R version 3.3.1 Patched (2016-10-08 r71472)
# system   x86_64, mingw32                            
# ui       RStudio (1.0.35)                           
# language (EN)                                       
# collate  English_Australia.1252                     
# tz       Australia/Hobart                           
# date     2016-10-11 

tsGRDC error

Dear Claudia,
please help resolving this issue:
`library(hddtools)

areaBox <- raster::extent(98.5, 105.5, 51.7, 56.5)

GRDC_catalogue_bratsk <- catalogueGRDC(areaBox = areaBox)

bratsk <- tsGRDC(stationID = GRDC_catalogue_bratsk$grdc_no, plotOption = TRUE)`

The code ends with an error:
`Using cached data.

Error in charToRaw(URL) : argument must be a character vector of length 1`

Thanks in advance!

Migrate CI

Migrate CI from travis-ci.org to GitHub Actions

Removing dependency from the rnrfa package

The rnrfa package is used here only to get coordinates of some hydrological stations. If the nrfa catalogue is saved within the package, the dependency can be remove safely as the coordinates are not expected to change in the future.

Error in graphics::par(old.par)

This issue is part of this ropensci review

@ledell said:

Found a possible bug:
library(raster)
Loading required package: sp
b <- brick('trmm_acc.tif')
plot(b)
Error in graphics::par(old.par) :
invalid value specified for graphical parameter "pin"

cran archived

@cvitolo Did you see that it was archived on cran? I imagine so, but jut in case i'm checking in.

A problem when trying to download some stations

I was trying to get mddPerYear of all the European stations. I have a problem when trying to download some stations, for example 6502110.
If:

tt <- tsGRDC(stationID = '6502110')
Station does not have monthly records

And then tt is empty, a NULL. But if I download the data from the ftp, i.e. the file europe.zip at ftp://ftp.bafg.de/pub/REFERATE/GRDC/ltdata/ that stations is not empty (here the file https://cl.ly/1H2I1Y2l3G06 )
I have tried to find an explanation on the documentation but nothing...can you help me?
UPDATE: looking in the code I can see that the function retrieves only the stations where the statistics field is 1, but I still don't understand...

Document more use cases

It would be great to add more examples of how this package could be used, e.g. in climate and hydrological research projects.

Cant download the TRMM data

I tried downloading the TRMM monthly data using the codes examples in the package,
but it showing the following error..

trying URL 'ftp://disc2.nascom.nasa.gov/data/TRMM/Gridded/3B43_V7/2009/3B43.090101.7.precipitation.accum'
trying URL 'ftp://disc2.nascom.nasa.gov/data/TRMM/Gridded/3B43_V7/2009/3B43.090201.7.precipitation.accum'
The following files are currently not available for download:

[1] "ftp://disc2.nascom.nasa.gov/data/TRMM/Gridded/3B43_V7/2009/3B43.090101.7.precipitation.accum"
[2] "ftp://disc2.nascom.nasa.gov/data/TRMM/Gridded/3B43_V7/2009/3B43.090201.7.precipitation.accum"
There are no files available, try a different temporal window.
Warning messages:
1: In download.file(url = filesURLs[j], destfile = file.path(td, basename(filesURLs[j])), :
InternetOpenUrl failed: 'The login request was denied
'
2: In download.file(url = filesURLs[j], destfile = file.path(td, basename(filesURLs[j])), :
InternetOpenUrl failed: 'The login request was denied

problem with TRMM on Windows and Linux

Happy to explore this, for now just the details.

On Windows.

TRMM(timeExtent = timeExtent, bbox = bbox)
latMax of bbox is out of the maximum extent [-50,50], new latMax of bbox is modified as follows: latMax = 50

Beware this function was tested on a unix machine. If you are not using a unix machine your results could be wrong. Please report any problem to [email protected], thanks!
Error in .local(.Object, ...) : 
  Unable to open ...3B43.*.
No error

Error in .rasterObjectFromFile(x, objecttype = "RasterBrick", ...) : 
  Cannot create a RasterLayer object from this file.

Is it possible to add a check if the GRDC server is down?

I cannot use the tool because apparently the server is not available:

> GRDC_catalogue_all <- catalogueGRDC(useCachedData = FALSE)
Retrieving data from data provider.
Error in download.file(theurl, temp, quiet = TRUE) : 
  cannot open URL 'ftp://ftp.bafg.de/pub/REFERATE/GRDC/catalogue/grdc_stations.zip'

In fact, the URL cannot be accessed. However, when you try to download data using tsGRDC the error is far less informative:

WolfeToneBridge <- tsGRDC(stationID = catalogueGRDC()$grdc_no[7126])
Using cached data.
Using cached data.
Error in download.file(zipFile, tf, quiet = TRUE) : 
  invalid 'url' argument

tsGRDC function error

Running the following

m3_Malawi<- tsGRDC(stationID = "1992700", plotOption = TRUE)

I receive the following error:

Using cached data. Station has monthly records Error in matrix(no_spaces, nrow = rows_to_read, byrow = TRUE) : data is too long

What does it mean?

GRDC catalogue

The currently GRDC catalogue is obsolete, it must be udated

Error: TRMM.vrt file does not exist

Hi,
Im trying to download the 3B42_v7 3 hourly data sing the TRMM function. The files get downloaded, however, I keep getting the following error:
"`D:\analysis_ravi\TRMM downlaod\kas_2016\TRMM.vrt' does not exist in the file system,
and is not recognised as a supported dataset name.

In addition: There were 17 warnings (use warnings() to see them)
Error in .rasterObjectFromFile(x, objecttype = "RasterBrick", ...) :
Cannot create a RasterLayer object from this file. (file does not exist)"

tried testing with 3B43_v7 dataset as well and same error.
Im using r 3.2.5 on windows 7 64-bit OS.

Any help would be greatly appreciated..

Regards
Manish
PhD Student
ATREE, India

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.