nationalparkservice / environmentalsetting_toolkit Goto Github PK

will vary depending on duration:
yly: 3% == 11 days
mly: 3% == 1 day

Update getGrids() and getGriddedMetrics() to indicate ACIS-derived vs. NPS PRISM-derived sourcing.

needed for summarized monthly and annual metrics

add test for values before passing to getDepartureCounts() and getRunCounts()

See PRISM web services for details

ACIS response returns string "no data available". Will retain this and add an sapply()-driven request and a nested list cleanup function to be called prior to writing to CSV.

Merge StationFinder, DataRequest, utilityfunctions

Function to read 800m data and crop with AOA polygons.

rList == error no data available stops execution

Intersect Polygons With Raster

Requires RGDAL. More info

Confirm logic and add getRunCounts() function.

Phone/email convo with Keith Eggleston (ACIS) Google Doc confirmed run logic

• Cannot specify run length, so need getRunCounts() function
• Can request values down to 0.01 if desired
• Do not need to specify add: mcnt or maxMissing since runs cannot have missing values

Decision: overload existing functions:
getWxObservations() - add metric
getGrids() - add metric and customBBox

No additional function needed.

Iterate using metric lookups; climateMetrics.Request == elem reduce

alter reduceFlags for CSP3 (micro-drought): reduceFlags <- c("date","mcnt","rmcnt")

could do a utility function: getProtocolStations()

SNAP Datasets - 771m

add NPSPRISMPaths and resolve reading in active folder

modify formatWxObservations() for nested missing values
at row # 444 (ack)

matrix(unlist(lapply(rList$data, "[", itemCount + 1)), ncol = 2, byrow = TRUE)[, 1]
where rList$data row like ["1957", ["29", "1957-09-06", 2, 0]],
instead of ["1957", ["11", 2]],

The date is messing with the unlist and lapply operations

Doc

c.f. Dave's comments

Add optional database connection when requesting protocol metrics

To replicate, request CST metric(s) using a set of UIDs that don't collect temperature data.

params = {
"uid": 31702,
"sdate": "1998-02-10",
"edate": "por",
"elems": [{
"name": "gdd",
"base": 32,
"interval": "yly",
"duration": "yly",
"reduce": [{
"reduce": "cnt_gt_0",
"add": "mcnt"
}],
"maxmissing": 1
}]

At line 1063

See /General ACIS web service.

Note that Climate Divisions are state-constrained

Include sapply() and remove lines from sample graphs.

Combine functions; departures consume a full date (%Y-%m%d) while runs consume %Y

generate metric value: X == above normal, -X == below normal

if request multiple climateParameters, a combined request is sent to ACIS; need to change to separate requests followed by 'merge' and addition of climateParameter to output dictionary

Add USHCN boolean to output data frame

see issue # 1

ACIS uses 50 as base; protocol specifies 32. See National Phenology Network info

Using the AOA feature service, request feature using unit code as GeoJSON. Transform to NAD83 geographic.

Example stations: 64477, 64559, 64451, 63500, 63514

pcpn_in_run_le_0.01 == NA

currently, returns 1 but should be 0

for function params
Docs

for metrics CST7, 8 and CSP 7, 8

Records with valid runs missing from output. Due to use of sapply() to request data and incorrect logic in getRunCounts() when decomposing nested column pcpn_in_run. Also, hiccuping may have occured due to erroneous check for countMissing element in ACIS response. Run requests do not return count missing since by definition a run cannot have a missing value.

Example affected uids: 67667, 77478, 79362, 69577

Debugging on 20180518 using lapply() instead of sapply() results in a list of data frames with consistent structure that can be bound into a single data frame and passed to getRunCounts(). Edits made to getStationMetrics() and cleanNestedLists() although the latter is not used when producing metric CSP3.

Resulting code:
metricSource <- lapply(climateStations, function(x){ getWxObservations( climateStations = x, climateParameters = cParam, sdate = sdate, edate = edate, duration = duration, interval = interval, reduceCodes = rCode, maxMissing = 10, metric = metric ) }) metricSourceCombo <- do.call(rbind, metricSource) metricSourceComboCleaned <- metricSourceCombo[metricSourceCombo$uid != "no data available",] metricData <- getRunCounts(rawCounts = metricSourceComboCleaned, runLength = 7, metric = metric) outputMetricFile(metricData, metric, filePathAndRootname)

Some records coming back with "no data available" even though the records have data in ACIS:
example uids: 44104, 44837(chunk 3). Others from chunk 4 (328 stations) and 5 (30) need additional QA checks and diagnosis. 360 stations total.