alshum / rwunderground Goto Github PK

View Code? Open in Web Editor NEW

77.0 12.0 14.0 745 KB

A simple R package to get historical and forecast weather data

R 100.00%

weather-data weather weather-underground weather-history

rwunderground's People

Contributors

Stargazers

Watchers

Forkers

akshayjh bnjcbsn erichare catherinecao2016 skeyrd shabbychef craigmohn jpshanno pssguy slentey wangdata 001101 xiaojunzeng jhanarato togettech

rwunderground's Issues

get around API limit calls for history

Get around API limit calls by using non-API URLs

Can't get it work

install.packages("devtools")
library(devtools)
install_github("ALShum/rwunderground")
rwunderground::set_api_key("...")
print(history(set_location(territory = "California", city = "San Francisco"), 20060405))

and I just got NULL.

What can I do wrong here?

Please add a test notebook with an example of work with the library, and not a pdf with a description of all the functions with which it is unclear how to work.

history_range() compatibility with dplyr 0.5.0

rbind_all() is deprecated. Please use bind_rows() instead.

history, history_range fail for location set as pws

the calls to history and history range fail, as the following shows:

history(location=set_location(PWS_id="KWASAMMA58"),date="20170101",key=get_api_key()))

The data returned from wunderground don't contain a result for visibility; this causes an empty result, num(0) to go into the list created on line 47-75 of history.R. This can be fixed by changing the statements of the form:
vis = as.numeric(x[[paste0("vis", suffix)]]),
to:
vis = ifelse(length(x[[paste0("vis", suffix)]])>0, as.numeric(x[[paste0("vis", suffix)]]), NA_real_)

dplyr bind_rows handle numeric NAs well, but chokes on variables that are all num(0)

date formats

support multiple date formats

support multiple features per call

wunderground.com returns extra data for some, but not all, stations on days when switch TO daylight saving time occurs

The resolution of the previous history daylight saving time edge case introduced a requirement that data requested from wunderground actually be from the day requested, or at least all be from the same day (local time). A look at interactive web results shows that sometimes the first observation from the next day is thrown in along with the data actually requested. The code resolving the earlier DST edge case catches and warns about this case, but I will change it today or tomorrow to fix problems with this extraneous data.

is_valid_airport function is not working on windows

library(rwunderground)
is_valid_airport('EISG')

Error in is_valid_airport("EISG") :
could not find function "is_valid_airport"

Time zones in hourly10day

The time is derived from the epoch with an origin at 1970-01-01 in the current time zone. This means that passing a tz argument applies the time zone offset twice, which leads to wrong timestamps.

use_metric should be set to false by default

lexical error: invalid char in json text.

lexical error: invalid char in json text.
<html class=""
(right here) ------^

How do I address this issue. I have installed rjson and RJSONIO.

What am I missing?

Sporadic error with history_daily()

a <-history_daily(location=set_location(lat_long="49.34,-123.04"), date = "20081017", use_metric = TRUE,
+               key = get_api_key(), raw = FALSE, message = TRUE)
[1] "Requesting: http://api.wunderground.com/api/3831f51632e13a0b/history_20081017/q/49.34,-123.04.json"
Error in hist$dailysummary[[1]] : subscript out of bounds

If I actually enter the above url, I get the json returned

in history_daily() dates a year either side i.e. date = "20071017" date = "20091017" work fine

your example for the history_range does not work

history_range(set_location(territory = "Hawaii", city = "Honolulu"), "20130101", "20130105")
[1] "Requesting: http://api.wunderground.com/api/9f1784685061b77e/history_20130101/q/Hawaii/Honolulu.json"
Error in (function (dt, year, month, yday, mday, wday, hour, minute, second, :
Invalid timezone of input vector: "Pacific/Honolulu"
[1] "Requesting: http://api.wunderground.com/api/9f1784685061b77e/history_20130102/q/Hawaii/Honolulu.json"
Error in (function (dt, year, month, yday, mday, wday, hour, minute, second, :
Invalid timezone of input vector: "Pacific/Honolulu"
[1] "Requesting: http://api.wunderground.com/api/9f1784685061b77e/history_20130103/q/Hawaii/Honolulu.json"
Error in (function (dt, year, month, yday, mday, wday, hour, minute, second, :
Invalid timezone of input vector: "Pacific/Honolulu"
[1] "Requesting: http://api.wunderground.com/api/9f1784685061b77e/history_20130104/q/Hawaii/Honolulu.json"
Error in (function (dt, year, month, yday, mday, wday, hour, minute, second, :
Invalid timezone of input vector: "Pacific/Honolulu"
[1] "Requesting: http://api.wunderground.com/api/9f1784685061b77e/history_20130105/q/Hawaii/Honolulu.json"
Error in (function (dt, year, month, yday, mday, wday, hour, minute, second, :
Invalid timezone of input vector: "Pacific/Honolulu"

A tibble: 0 x 0

error in conditions() with parsing temp?

In data.frame(weather = cond$weather, temp = as.numeric(cond[[paste0("temp_",  :
  NAs introduced by coercion

I gues the NA's are not correclty parsed?

however: when I do a raw request

conditions <- rwunderground::conditions(location = "pws:ILEIDEN46", use_metric = TRUE, raw = TRUE)
as.numeric(conditions$current_observation$temp_c)  # does work. 

conditions$current_observation[[temp_c]]  does not work.

use dplyr bind_col and bind_row instead of do.call(cbind ...) and do.call(rbind ...)

Error in (function (..., row.names = NULL, check.rows = FALSE, check.names = TRUE, : arguments imply differing number of rows: 1, 0

When trying to obtain historic weather data from personal/private weather stations using history_range() and set_location(PWS_id =" ") this error is returned.

Example:
history_range(set_location(PWS_id = "KORPORTL775"), date_start = "20170520", date_end = "20170521")

Error in history_daily label:bug

Receiving this error message when running history_daily for a location with Precipitation value returned.

Warning message:
In do.call(paste0("as.", class), list(val)) : NAs introduced by coercion

Example: Postal Code = 60666, Date = 4-24-2018

Sample r:

API <-'XXXXX'
Location <- '60666'
rwunderground::set_api_key(API)
location <-rwunderground::set_location(zip_code = Location)
Date = '20180424'
h<-rwunderground::history_daily(location, Date,key=rwunderground::get_api_key(), message = FALSE)

Thanks!

better documentation for lat_long

I tried with an array of doubles, but that failed. More specific documentation would have helped. I will submit a PR for this.

What is the plan for the no_api option in history_range?

Here is how it looks currently:

rwunderground/R/history.R

Lines 263 to 265 in 54b504b

    
           if (no_api) { 
        
             warning("no_api: this feature is not yet working") 
        
           }

current_hurricane has parsing issues

FR: A hybrid of history_daily and history_range

Currently, history_range() returns a number of observations over the course of the days specified by date_start and date_end.

I'm only really interested in one overall observation for a range of days over the calendar year, so I'd be more inclined to use history_daily(), except that I have to iterate through each of the days of the year. That in itself is not a problem, but I like that that history_range() provides a limit to the API key to avoid running over their 10 requests/minute limit. Would it be possible to offer this as an alternative?

Error on forecast10day function

Receiving this error when using the function:

Error: 'as_tibble' is not an exported object from 'namespace:dplyr'

Passing raw=TRUE removes the error but returns a file type that is not nicely parsed.

-9999, -999, -99 should be encoded as NA

countrycode dependency breakage

Hi,

I am the maintainer of the countrycode package which you use as a dependency.

We're moving to version 1.00.0, and changing the name of the object where regexes and country codes are stored. Before, you had to look into the countrycode data.frame. Now it's called codelist.

You'll probably want to add a line to check package version before digging right into that object.

This change is permanent; you can rest assured that countrycode will be a reliable dependency into the future.

Sorry for the trouble.

http://github.com/vincentarelbundock/countrycode

history() and history_range() return NA values but raw JSON does not

I'm using rwunderground 0.1.6, R 3.4.2 on Windows 7. This code:

h1 <- history(set_location(PWS_id = "INEWSOUT879"), date = 20170103, use_metric = TRUE)

returns a data frame where the values for viz and for all variables after the pressure column are NA.

However, the raw JSON for the same request:

http://api.wunderground.com/api/261d580a0cf44286/history_20170103/q/pws:INEWSOUT879.json

returns values for these variables. For example, the last observation for precipm is 4.3.

I see the same issue using history_range(), but not using history_daily().

Output from sessionInfo():

R version 3.4.2 (2017-09-28)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

Matrix products: default

locale:
[1] LC_COLLATE=English_Australia.1252  LC_CTYPE=English_Australia.1252    LC_MONETARY=English_Australia.1252
[4] LC_NUMERIC=C                       LC_TIME=English_Australia.1252    

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] rwunderground_0.1.6 lubridate_1.6.0     rvest_0.3.2         xml2_1.1.1          pander_0.6.1       
 [6] knitr_1.17          cluster_2.0.6       factoextra_1.0.5    dtwclust_4.1.1      dtw_1.18-1         
[11] clue_0.3-54         proxy_0.4-17        zoo_1.8-0           odbc_1.1.3          DBI_0.7            
[16] dplyr_0.7.4         purrr_0.2.3         readr_1.1.1         tidyr_0.7.1         tibble_1.3.4       
[21] tidyverse_1.1.1     extrafont_0.17      bindrcpp_0.2        ggraph_1.0.0        ggplot2_2.2.1      
[26] igraph_1.1.2       

loaded via a namespace (and not attached):
 [1] nlme_3.1-131        bit64_0.9-7         httr_1.3.1          rprojroot_1.2       backports_1.1.1    
 [6] tools_3.4.2         R6_2.2.2            lazyeval_0.2.0      colorspace_1.3-2    tidyselect_0.2.0   
[11] gridExtra_2.3       mnormt_1.5-5        curl_3.0            bit_1.1-12          compiler_3.4.2     
[16] extrafontdb_1.0     flexclust_1.3-4     pkgmaker_0.22       labeling_0.3        scales_0.5.0       
[21] psych_1.7.8         stringr_1.2.0       digest_0.6.12       foreign_0.8-69      rmarkdown_1.6      
[26] pkgconfig_2.0.1     htmltools_0.3.6     highr_0.6           rlang_0.1.2         readxl_1.0.0       
[31] bindr_0.1           jsonlite_1.5        magrittr_1.5        modeltools_0.2-21   Matrix_1.2-11      
[36] Rcpp_0.12.13        munsell_0.4.3       viridis_0.4.0       stringi_1.1.5       yaml_2.1.14        
[41] MASS_7.3-47         plyr_1.8.4          grid_3.4.2          blob_1.1.0          ggrepel_0.7.0      
[46] bigmemory.sri_0.1.3 forcats_0.2.0       udunits2_0.13       lattice_0.20-35     haven_1.1.0        
[51] hms_0.3             ggpubr_0.1.5        rngtools_1.2.4      reshape2_1.4.2      codetools_0.2-15   
[56] stats4_3.4.2        XML_3.98-1.9        glue_1.1.1          evaluate_0.10.1     modelr_0.1.1       
[61] selectr_0.3-1       tweenr_0.1.5        foreach_1.4.3       Rttf2pt1_1.3.4      cellranger_1.1.0   
[66] gtable_0.2.0        assertthat_0.2.0    ggforce_0.1.1       xtable_1.8-2        broom_0.4.2        
[71] RSpectra_0.12-0     rsconnect_0.8.5     viridisLite_0.2.0   iterators_1.0.8     registry_0.3       
[76] units_0.4-6         bigmemory_4.5.19

Make set_location less restrictive

Setting location strings is needlessly restrictive in a lot of edge cases.

No more free-tier for API key

Just an FYI, since I went to get an API key to use your package this morning:

To improve our services and enhance our relationship with our users, we will no longer provide free weather API keys as part of our program. If you have been directed to download our Weather Underground free API key by a third party provider, please contact your vendor for resolution.

https://www.wunderground.com/weather/api

Why cant I have WebCam urls for a specific date ?

Can i have webcam urls for specific Location for a specific date ?

support multiple locations per call and date ranges per call

End of Service for WU API

Weather Underground API's End of Service Announcement

It seems like the only way to access WU data moving forward is through The Weather Company's API 👎.

Do you plan to update the rwunderground package to accommodate for this shift?

daylight saving time

The following example shows a problem with the way the text value of time returned from wunderground is processed in the hour when daylight saving time ends. Recall that time advances from 01:59:59 daylight time to 01:00:00 standard time, at least in Seattle. Passing the time 01:53:00 to as.POSIXct() is could refer to two separate instants an hour apart, and apparently yields the second such instant to occur (01:53 standard time).

df <- rwunderground::history_range(location="KBFI",date_start="20091101",
date_end="20091101",key=get_api_key())
head(df$date)
[1] "2009-11-01 00:53:00 PDT" "2009-11-01 01:00:00 PDT" "2009-11-01 01:26:00 PST" "2009-11-01 01:53:00 PST"
[5] "2009-11-01 01:53:00 PST" "2009-11-01 02:53:00 PST"

The 3rd and 4th results should have been PDT not PST.

It appears that wunderground returns results in chronological order, as the following screenshot of the web result with METARS data included shows:

I haven't looked down a level to see if the calls to wunderground return the METARS data with UTC times, but as long as there as the times are assumed to be increasing in real (UTC) terms, it would be easy to determine when the switch from daylight to standard occurs based on the one nonincreasing timestamp we expect. If there are zero or two, decisions need to be made.

It occurs to me that the details of which (PDT/PDT in this example) timezone is chosen may be windows-dependent, since timezone functions rely heavily on the operating system, IIRC. I have no convenient way to test on a non-windows platform. However, the problem exists on all systems, since location and local time is not an unambiguous specification of an instant in time during this one hour each year.

	if (no_api) {
	warning("no_api: this feature is not yet working")
	}