stateofcalifornia / calcat Goto Github PK

To fulfill the stated objectives of encouraging reuse in other jurisdictions, I recommend adding an explicit open source license to this repo. The MIT License is the most permissive to encourage collaboration and reuse, if so inclined.

https://opensource.org/licenses/MIT

When I do run all for al_cat_data_routine.rmd, I get the following error on the MOBS section:

mobs <- grab_mobs()
Error: lexical error: invalid char in json text.
saved_old_versions/mobs_New Yor
(right here) ------^

Note: the arrow should be under the a in saved

I highly recommend adding a code testing framework and tests to this package.

When running the script for California, I found that the code expected to convert the timeseries JSON element into a dataframe but for 5 of the counties the timeseries element is empty and so the code fails. These counties are:
'06003', '06063', '06105', '06091', '06049'

Note: The actual error message I got was that mutate failed on as.Date. No surprise since it was trying to act on the "date" column and there was no such column!

P.S. It failed getting the OBSERVED intervention. The strong and weak intervention data might be OK.

We want to gather the community of people working on CalCAT this week. Please register here bit.ly/calcatreg
@adamr-hub
@adannenberg
@AK-MCH-EPI
@anielsen001
@aschl
@baajur
@Bartolac
@briviere
@charada916
@chasemc
@CJE0113
@ClauGISCorona
@colearendt
@DaveParr
@DevenLu
@djd2164
@dnasko
@DrOlivia
@fersalme
@FloatingRootBeer
@GisKid
@guillermodeandajauregui
@hchawla-1
@jasonajones73
@jbayham
@jdh--
@jeffokamoto
@Jenniferha014
@joelrip
@jonnijohnson
@Josteniok
@jpflores-13
@jwilliamsholt
@karlanderson
@kjanevski
@kuanhulio
@lakikowolfe
@LaurenHarriman
@leetcode-notes
@leirbagflow
@MAYHEM-Lab
@mccorvie
@mgermino
@mshigekawa
@natedonnelly
@natekotila
@naterajj
@nayefahmad
@nerfherder987
@nkgithub01
@nturaga
@PaulSaam
@PrincessG
@Qberto
@qizongjun
@rashamad
@rdavidp
@rdavidparker
@restartus
@rkiddy
@sailing0505
@samedelstein
@scallionpancakes
@shalziekay
@skybett
@smach
@srravula1
@StevenLassen
@taqeric
@tdatu
@TJB8526
@tkleykamp
@unchainedbite
@vdtobias
@wehrley
@weikang9009
@WheresHJ

In the grab_epi_forecasts.R file and line 106 of the Rmd file, the get_epi_forecasts function was throwing an error, saying the URL was bad. I updated to https://raw.githubusercontent.com/epiforecasts/covid-regional/master/united-states/regional-summary/rt.csv and it seems to have resolved the issue

I get the following error:

Error: Problem with mutate() input date.
x do not know how to convert 'date' to class “Date”
ℹ Input date is as.Date(date).

The issue is happening in cal_cal_data_routine.rmd:

74 can_county_observed <- grab_can_county_observed_intervention()
fwrite(can_county_observed, paste0(data_path,"can_full_reff_table.csv"))

two files read in for ICL but only one is now needed, causing odd display of the statewide nowcast map

I'm looking for the following datasets to run the Shiny App.

https://data.chhs.ca.gov/dataset/6882c390-b2d7-4b9a-aefa-2068cee63e47/resource/6cd8d424-dfaa-4bdd-9410-a3d656e1176e/download/covid19data.csv in read_hhs_covid.R

https://raw.githubusercontent.com/nytimes/covid-19-data/master/us-counties.csv and https://raw.githubusercontent.com/CDPHrusers/List-of-US-States/master/states.csv in get_counties.R

Hello, I recently discovered this tool (very cool!) and have just started working with it to see if I can run it with data from Washington state. I've created a fork in the branch and input "Washington" where instructed, but am running into an issue running the cal_cat_data_routine markdown file.

Specifically, in the grab_can_state_observed_intervention formula I'm unable to import the data from data.covidactnow.org (can.ca <- jsonlite::fromJSON(paste0("https://data.covidactnow.org/latest/us/states/",get_state_abbrv(State),".OBSERVED_INTERVENTION.timeseries.json"))$timeseries).
The result is an empty data.frame. Do I need to set up an API with them in order to access these data?

Also as a side question, is there code in your github to recreate your ensemble models for Flu?

Thanks,
Katrin from Public Health Seattle & King County

#15

There is some code that contains helpful links to repos that contain the data used in visualization, but some are missing. Would anyone be able to point to where they are; e.g. Johns Hopkins, LEMMA, Stanford? Alternatively, having a source for the original data would be great. Thanks!

I'm a member of the ensemble contributors for the weekly CDC death projections and would like to add my model projections to this site. I'm looking for information on the format you want for the data but can't find a description. I've looked at some of the contributed data and there seems to be no consistency to the format of the data and none of the submissions seems similar to the format used for the CDC data. It looks like everyone is using whatever they want. I don't understand how you can create an ensemble forecast without a format for contributed data. Can you explain what you would like to see?

The grab_ihme.R file refers to Hospitalization_all_locs.csv within the ihme-covid19.zip file. It looks like this should actually be Reference_hospitalization_all_locs.csv.

I have not seen anything newer than 2 months ago on this site. Is it no longer in use?

FYI I received this error message:

Error: Problem with `mutate()` input `date`.
x do not know how to convert 'date' to class “Date”
i Input `date` is `as.Date(date)`.

on line 74 of cal_cat_data_routine.Rmd:

can_county_observed <- grab_can_county_observed_intervention()

That seems to be because all the loop_can_cnty_observed() results for Massachusetts counties are coming back FALSE so I have no data. For example, valid <- RCurl::url.exists("25017") is returning FALSE on my system. However, if I run

cnty <- jsonlite::fromJSON(url)$timeseries %>% as.data.frame() %>% mutate(date = as.Date(date))
    cnty <- mutate(cnty, fips_char = fips_code, fips = as.numeric(fips_code))

I do get data. So I'm not sure why the valid check is coming back FALSE, although I suspect it has something to do with my system setup. I'm on Windows 10 (I know, I know), R 4.0.0, RCurl 1.98-1.2.

Working through the code to apply to Alaska and noting an issue with the grab_can_county_observed_intervention function. This appears to stem from the loop_can_cnty_observed function where the can.ca.co object returns a null. I can hard code this by specifying each fips code explicitly but the lapply function keeps failing.

Will work with it a bit to come up with my own solution but also reaching out to see if this is a simple oversight on my part.

upon further inspection it appears to be related to the RCurl::url.exists(url) code in the loop function returning false for all values.

Error in get_counties() : could not find function "get_counties"

Pardon me if this is a newbie question, and I just don't know where to look.

Will the list of county-level and CA-level statistics eventually be updated daily?
It looks like they have not been updated since June 10.
https://github.com/StateOfCalifornia/CalCAT/tree/master/data/CA

The data tables found there extend beyond that date, but it looks like those are projections.
can_state_reff_table.csv
can_full_reff_table.csv

When the initial script to download data is run: it returns the following:

ihme <- grab_ihme()
trying URL 'https://ihmecovid19storage.blob.core.windows.net/latest/ihme-covid19.zip'
Content type 'application/octet-stream' length 19866543 bytes (18.9 MB)
==================================================
downloaded 18.9 MB

Error in fread(file.path(temp2, paste0(folder, "/Hospitalization_all_locs.csv"))) :
File '/var/folders/kv/nt1xs5dd5637lbxxgg4z40n00000gn/T//RtmpXq2wsJ/file136c8464f30bd/2020_06_24.02/Hospitalization_all_locs.csv' does not exist or is non-readable. getwd()=='/Users/jasonpeskin/Desktop/CalCAT'

Just tried to run global.R and need to install a few packages first. I recommend updating to something like the following for ease of use:

If a package is installed, it will be loaded. If any
are not, the missing package(s) will be installed
from CRAN and then loaded.

First specify the packages of interest
packages = c("tidyverse", "geomorph",
"phytools", "viridis")

Now load or install&load all

package.check <- lapply(
packages,
FUN = function(x) {
if (!require(x, character.only = TRUE)) {
install.packages(x, dependencies = TRUE)
library(x, character.only = TRUE)
}
}
)

https://vbaliga.github.io/verify-that-r-packages-are-installed-and-loaded/

Highlight for nowcast map is red, which is confusing given that R-eff color scheme for the map has red as the highest R-eff category. Shift to heavy outline to indicate county is selected?

You don't notice this on desktop, but on phone it's weird.
The , "nowcast" "forecast" "scenario" words all separated from icons. They are not clickable.

this is a test @mccorvie

both ygg.csv and ygg_county.csv aren't available in the current data files, are these referencing yuganggu.csv and yuganggu_county.csv?