Shows original column names, weathercan column names and units. Possibly also definitions from http://climate.weather.gc.ca/glossary_e.html ?

There is an unnecessary "is" in the first sentence in the README file.

This package is makes it easier to search for and download multiple months/years of historical weather data from Environment and Climate Change Canada (ECCC) website.

i.e. filter_, etc.

Not for upcoming CRAN submission. I'm uncertain how to ultimately implement this as some user might not like it. Perhaps just as a listcol = TRUE argument in weather().

A way of dissemination data from ECCC:

https://dd.meteo.gc.ca/climate/doc/observations/README_climate.txt

With the data being here:

https://dd.meteo.gc.ca/climate/observations/

It would require a bit of work under the hood but it might at least even be worth a speed test. Haven't fully explored the datamart for this product so it may not be fully appropriate.

In advance of a CRAN submission I think it is prudent to have a good vignette built. The README is basically there but I think we could highlight a few more of the intricacies. Particularly I think we could flesh out the time interval argument explanation. That wasn't immediately obvious to me.

A fairly well developed package already exists over at https://github.com/gavinsimpson/canadaHCD

First of, thank you for creating this package.

Now, about the error: I am trying to run the following code:

library(weathercan)
> kam <- weather_dl(station_ids = 51423, start = "2016-01-01", end = "2016-02-15")
Error in if (tz <= 0) tz <- paste0("+", abs(tz)) else tz <- paste0("-",  : 
  argument is of length zero

Here's the traceback:

Error in if (tz <= 0) tz <- paste0("+", abs(tz)) else tz <- paste0("-", : argument is of length zero
 3. tz_calc(coords = unique(preamble[, c("lat", "lon")]), etc = TRUE)
 2. weather_format(w = w, interval = interval, tz_disp = tz_disp, string_as = string_as, quiet = quiet, preamble = preamble)
 1. weather_dl(station_ids = 51423, start = "2016-01-01", end = "2016-02-15") 

I have tried different station_ids and start & end but get the same error.

Have Travis CI rebuild the package at intervals to catch any problems that occur if Environment and Climate Change Canada make changes to their side of things.

• Add climate normals data base
• Add ability to update climate normals data frame
• Add function to extract climate normals for a given station? (would pretty much just be a dplyr filter wrapper)
• ftp://[email protected]/Pub/Normals/English/

OR

Implement a normals calculation

• Definitely would be faster and would require less bundled data
• Possibly more prone to errors?

Add basic information to the man page for this item

Add progress message to weather_html() under verbose conditions. This will act like a progress indicator for large downloads, so users can see that things are progressing.

• Need an argument so that weather() doesn't always have output.
• A progress bar might be useful though I'm not totally clear how that interact with web scraping

Expected Behavior

weather_dl() should always return a data frame.

Current Behavior

Sometimes, weather_dl() reports

Error in read.table(file = file, header = header, sep = sep, quote = quote,   : 
more columns than column names

Steps to Reproduce (for bugs)

1. When extracting data over a date range that includes 2018-03-31 through 2018-04-01, the error is produced. This is not weather-station dependent.

kam <- weather_dl(station_ids = 51423, start = "2018-03-31", end = "2018-04-01")
kam2 <- weather_dl(station_ids=42203, start = "2018-03-31", end = "2018-04-01")
kempt <- weather_dl(station_ids = 27534, start = "2018-03-31", end = "2018-04-01")

Any other date range is ok:
kam <- weather_dl(station_ids = 51423, start = "2018-03-01", end = "2018-03-31")
kam <- weather_dl(station_ids = 51423, start = "2018-04-01", end = "2018-04-30")
kam <- weather_dl(station_ids = 51423, start = "2017-03-31", end = "2017-04-01")

Possible Solution

Comparing the column names of both dates retrieved individually (2018-03-31, 2018-04-01) returns no differences with testthat::expect_equivalent().

Your Environment

Session info -------------------------------------------------------------------------------------------------------------
setting value
version R version 3.5.0 (2018-04-23)
system x86_64, mingw32
ui RStudio (1.1.383)
language (EN)
collate English_Canada.1252
tz America/New_York
date 2018-05-29

Packages -----------------------------------------------------------------------------------------------------------------
package * version date source
assertthat 0.2.0 2017-04-11 CRAN (R 3.5.0)
base * 3.5.0 2018-04-23 local
bindr 0.1.1 2018-03-13 CRAN (R 3.5.0)
bindrcpp * 0.2.2 2018-03-29 CRAN (R 3.5.0)
compiler 3.5.0 2018-04-23 local
curl 3.2 2018-03-28 CRAN (R 3.5.0)
datasets * 3.5.0 2018-04-23 local
devtools 1.13.5 2018-02-18 CRAN (R 3.5.0)
digest 0.6.15 2018-01-28 CRAN (R 3.5.0)
dplyr 0.7.5 2018-05-19 CRAN (R 3.5.0)
glue 1.2.0 2017-10-29 CRAN (R 3.5.0)
graphics * 3.5.0 2018-04-23 local
grDevices * 3.5.0 2018-04-23 local
httr 1.3.1 2017-08-20 CRAN (R 3.5.0)
lubridate 1.7.4 2018-04-11 CRAN (R 3.5.0)
magrittr 1.5 2014-11-22 CRAN (R 3.5.0)
memoise 1.1.0 2017-04-21 CRAN (R 3.5.0)
methods * 3.5.0 2018-04-23 local
pillar 1.2.1 2018-02-27 CRAN (R 3.5.0)
pkgconfig 2.0.1 2017-03-21 CRAN (R 3.5.0)
purrr 0.2.4 2017-10-18 CRAN (R 3.5.0)
R6 2.2.2 2017-06-17 CRAN (R 3.5.0)
Rcpp 0.12.16 2018-03-13 CRAN (R 3.5.0)
rlang 0.2.0 2018-02-20 CRAN (R 3.5.0)
stats * 3.5.0 2018-04-23 local
stringi 1.1.7 2018-03-12 CRAN (R 3.5.0)
stringr 1.3.1 2018-05-10 CRAN (R 3.5.0)
testthat 2.0.0 2017-12-13 CRAN (R 3.5.0)
tibble 1.4.2 2018-01-22 CRAN (R 3.5.0)
tidyr 0.8.1 2018-05-18 CRAN (R 3.5.0)
tidyselect 0.2.4 2018-02-26 CRAN (R 3.5.0)
tools 3.5.0 2018-04-23 local
utils * 3.5.0 2018-04-23 local
weathercan * 0.2.6 2018-05-29 Github (3b8d291)
withr 2.1.2 2018-03-15 CRAN (R 3.5.0)
xml2 1.2.0 2018-01-24 CRAN (R 3.5.0)
yaml 2.1.18 2018-03-08 CRAN (R 3.5.0)

The R version of weathercan is likely too restrictive. Using this app, shows that we likely only need a version of 3.2.0. Even to be conservative we may considered only requiring 3.3.0 so that older versions of R can run weathercan.

Allow the options of converting times to 'relative' time. I.e. if multiple timeszones, force each timezone to UTC (do not convert). This means that 9am in TZ1 corresponds to 9am UTC, also 9am in TZ2 corresponds to 9am UTC. This way daily patterns are directly comparable, but the timezone attribute is a dummy variable.

I think that we might consider a different package name for a couple reasons:

• Environment Canada might object to us calling it by their name
• It isn't totally descriptive of what the package does
• It doesn't encompass all the other types of environmental monitoring that ec does.

Some other suggestions:

• canweatheR
• canadaweatheR
• canmeteoR

From goodpractices::gp(): Do not import packages as a whole, as this can cause name clashes between the imported packages. Instead, import only the specific functions you need.

Hi, lots of duplication on the 'rclimateca' package. Haven't updated it for a while given PhD school constraints, but feel free to PR missing features that are available here that are not currently available in rclimateca. Cheers!

This is note to remind myself to run these tests on weathercan:

http://www.masalmon.eu/2017/06/17/automatictools/

When no stations within the distance (dist) specified, the message doesn't make sense.

The flags vignette makes a reference to a "missing data" flag, which is no longer present. Choose a new example and update the vignette.

weathercan was selected for RViews July 2018: Top 40 New Packages https://rviews.rstudio.com/2018/08/27/july-2018-top-40-new-packages/ 🎉

This is a problem and short-term solution reported by John Lewis.

Question: Is it possible to selectively download only winter months across multiple stations and multiple years?
Answer: It is possible in the future to implement that for hourly data, but it is currently not implemented (and won't be immediately implemented)

John presented this work-around in the meantime:

library("weathercan")
library("tidyverse")
stations_search("Ottawa", interval = "hour")

#example of downloading data for specific time period(s) during a year
#in this case it is for 4 years of winter hourly data for Ottawa
st <- c("2011-12-14","2012-11-01","2013-11-01","2014-11-01")
ed <-c("2012-03-31","2013-03-31","2014-03-31","2015-03-31")
otta <- vector("list", length(st))
for (i in seq_along(st)){
    otta[[i]] <- weather_dl(station_id=49568,start = st[[i]],
       end = ed[[i]])
}

otta

##########################
#View(out)
#if you want to combine the data into one large data frame you can
#save the output in a list, then use dplyr::bind_rows(output) to
#combine the output into a single data frame.
ott <- dplyr::bind_rows(otta)
head(ott)
tail(ott)

For instance this:

Can be changed to this:

  ## Convert to tibbles for consistency
  data <- dplyr::as_tibble(data)
  weather <- dplyr::as_tibble(weather)

FWIW I realize that it was me who did this in the first place 🤦‍♂️

Quality column shows incorrectly encoded characters

Hi,

I recently discovered your extremely useful package. Thank you for creating it!

I was wondering how often the stations dataframe is updated, since I noticed a discrepancy between it and weather.gc.ca. It's annoying how weather stations stop reporting/change on the website from time to time, but is there a way to get a refresh of the list so we can manually fix any scripts whenever a change occurs?

Thanks!

To avoid NOTES on CRAN check which detect pseudo unassigned variables resulting from NSE

Expected Behaviour

weather_dl(station_id = 49568, interval = "day", start = "2012-11-01", end = "2013-04-05")

Should return daily data from 2012-11-01 to 2013-04-05

Current Behaviour

Data is truncated at 2012-12-31

We should consider adding a memoise ability to any download function.

This cache's the output in a session so that unless the parameters have changed, identical download calls will only be downloaded once per session. It is fairly easy to implement and can provide a reasonable buffer to repeated identical downloads.

Here is the memoise package: https://github.com/r-lib/memoise

Check tests and make sure that they're CRAN friendly, if not, add skip on CRAN

Figure out a way for users to update internal stations data frame. They can download a new stations data frame, but it doesn't update the internal one

The overhead for these two lines is pretty significant:

https://github.com/steffilazerte/weathercan/blob/a5948a687df58b65657a3b0e61a813812d4095d3/R/stations.R#L190-L191

Particularly for mac and linux this requires installing quite a lot of other material. We should try to remove it.

Look into adding French as a language option for messages, errors, and, possibly, documentation.

I was wondering if there is any way to got lat/lon coordinates at a higher precision than 2 decimal places, which is an error of plus or minus 1.1km or less.

Thank you, this package has proven invaluable to our research team.

J

Expected Behavior

weather_dl should return weather data

Current Behavior

weather_dl errors out

Steps to Reproduce (for bugs)

Using GitHub PAT from envvar GITHUB_PAT
Downloading GitHub repo ropensci/weathercan@master
from URL https://api.github.com/repos/ropensci/weathercan/zipball/master
Installing weathercan
'/Library/Frameworks/R.framework/Resources/bin/R' --no-site-file --no-environ --no-save  \
  --no-restore --quiet CMD INSTALL  \
  '/private/var/folders/48/q6ltldjs251000_wvjrdy_vm0000gn/T/Rtmp2j8c0e/devtoolse0ba9742238/ropensci-weathercan-0eafb91'  \
  --library='/Library/Frameworks/R.framework/Versions/3.5/Resources/library' --install-tests 

* installing *source* package ‘weathercan’ ...
** R
** data
*** moving datasets to lazyload DB
** inst
** tests
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
* DONE (weathercan)
> library(weathercan)
> kam.pg <- weather_dl(station_ids = c(48248, 51423),
+                       start = "2016-01-01", end = "2016-02-15")
Error in names(w) <- n : 
  'names' attribute [25] must be the same length as the vector [24]

Your Environment

> devtools::session_info()
Session info -----------------------------------------------------------------------------------
 setting  value                       
 version  R version 3.5.0 (2018-04-23)
 system   x86_64, darwin15.6.0        
 ui       RStudio (1.1.442)           
 language (EN)                        
 collate  en_CA.UTF-8                 
 tz       America/Vancouver           
 date     2018-05-10                  

Packages ---------------------------------------------------------------------------------------
 package    * version date       source                              
 assertthat   0.2.0   2017-04-11 CRAN (R 3.5.0)                      
 base       * 3.5.0   2018-04-24 local                               
 bindr        0.1.1   2018-03-13 CRAN (R 3.5.0)                      
 bindrcpp   * 0.2.2   2018-03-29 CRAN (R 3.5.0)                      
 compiler     3.5.0   2018-04-24 local                               
 curl         3.2     2018-03-28 CRAN (R 3.5.0)                      
 datasets   * 3.5.0   2018-04-24 local                               
 devtools     1.13.5  2018-02-18 CRAN (R 3.5.0)                      
 digest       0.6.15  2018-01-28 CRAN (R 3.5.0)                      
 dplyr        0.7.4   2017-09-28 CRAN (R 3.5.0)                      
 git2r        0.21.0  2018-01-04 CRAN (R 3.5.0)                      
 glue         1.2.0   2017-10-29 CRAN (R 3.5.0)                      
 graphics   * 3.5.0   2018-04-24 local                               
 grDevices  * 3.5.0   2018-04-24 local                               
 httr         1.3.1   2017-08-20 CRAN (R 3.5.0)                      
 knitr        1.20    2018-02-20 CRAN (R 3.5.0)                      
 lubridate    1.7.4   2018-04-11 CRAN (R 3.5.0)                      
 magrittr     1.5     2014-11-22 CRAN (R 3.5.0)                      
 memoise      1.1.0   2017-04-21 CRAN (R 3.5.0)                      
 methods    * 3.5.0   2018-04-24 local                               
 pillar       1.2.2   2018-04-26 CRAN (R 3.5.0)                      
 pkgconfig    2.0.1   2017-03-21 CRAN (R 3.5.0)                      
 purrr        0.2.4   2017-10-18 CRAN (R 3.5.0)                      
 R6           2.2.2   2017-06-17 CRAN (R 3.5.0)                      
 Rcpp         0.12.16 2018-03-13 CRAN (R 3.5.0)                      
 rlang        0.2.0   2018-02-20 CRAN (R 3.5.0)                      
 stats      * 3.5.0   2018-04-24 local                               
 stringi      1.2.2   2018-05-02 CRAN (R 3.5.0)                      
 stringr      1.3.0   2018-02-19 CRAN (R 3.5.0)                      
 tibble       1.4.2   2018-01-22 CRAN (R 3.5.0)                      
 tidyr        0.8.0   2018-01-29 CRAN (R 3.5.0)                      
 tidyselect   0.2.4   2018-02-26 CRAN (R 3.5.0)                      
 tools        3.5.0   2018-04-24 local                               
 utils      * 3.5.0   2018-04-24 local                               
 weathercan * 0.2.5   2018-05-10 Github (ropensci/weathercan@0eafb91)
 withr        2.1.2   2018-03-15 CRAN (R 3.5.0)                      
 yaml         2.1.19  2018-05-01 CRAN (R 3.5.0)    

I think we should test this with all the changes in dplyr 0.70 especially the tidyeval programming stuff. And I think we should do this ahead of a cran submission. Of course this means learning all that new stuff but it probably has to happen at some point anyway.