mazamascience / airsensordataviewer Goto Github PK

The time slider seems to exhibit glitches occasionally causing the slider to lay on top of the timeseries – it may occur when you hide and unhide the timeseries several times (there is an image of it below, but unfortunately I don’t have a clear reproducible example).

(Version: 1.0.1)

Further clarification regarding the time zone throughout the tool would be helpful, time zones should be labeled where appropriate and it would be great if visual alert could be added, such as a popup window, that would appear if the user’s date selection bridges one of the two time changes (e.g., “FYI: your selected date range includes one of the daylight savings time changes”).

(versions 1.0.1)

New videos are being created at:

http://data.mazamascience.com/PurpleAir/v1/videos/2020/

We probably need to split these out into per-month subdirectories and then reference them in the DataViewer. That is a task for Jon. Jon also needs to create a script to populate an archive with videos for previous days. Overall tasks include:

• 1) Jon rewrites script to place videos in per-month subdirectories
• 2) Hans updates DataViewer to point to these new videos
• 3) Jon writes script to populate the archive

Bar Chart requested updates

• Remove transparency on bar chart
• Need larger margins on plot
  a. Space between conc and y-axis
• Move sensor name to upper left --- make bar chart narrower to allow for sensor name in top left
• Extend y-axis to top of white box

Historical Data Page - Tabular Data Tab

• All data in columns should be centered
• Top bar Text changes
  • “Sensor” change to “Sensor Name”
  • “Sensor.Type” change to “Sensor Model” and text “PA-II (PurpleAir)”
• Include option to download tabular data shown on this tab
• Remove search box feature – not needed

Not all community videos generated for communities in September. This is specific for Seal Beach, but may impact other communities. Timelapse not working for SealBeach community. When viewing videos for September, only the 9/30 date is present for a video.

http://data.mazamascience.com/PurpleAir/v1/

http://tools.mazamascience.com:6709/asdv/test/?_inputs_&main_panel_ui_1-sensor_select=%22SCSB_18%22&main_panel_ui_1-past_select=%227%22&main_panel_ui_1-community_select=%22Seal%20Beach%22&main_panel_ui_1-date_select=%222020-09-17%22&tab=%22video%22

About Page requested updates:

• Update text and add link to paper - https://doi.org/10.1016/j.envsoft.2020.104832
• Replace references to AirShiny with DataViewer
• Add color scale with concentration breaks information to the About page

The time series map does not filter the points on community selection

Historical Data Page - Overview Tab - Slider updates

Slider should not overlay over bar chart
Options:

1. Option to move this to be the axis of the BarChart to create a combined axis overlapping
2. Option 2 move slider to be right above barchart

The AirSensorDataViewer package is using the {golem} framework. Many of the functions currently in the package do not contain function and parameter descriptions. The package is documented just like any other roxygen2 documentation and R package.

There are a occasional visual glitches when using the show/hide functionality on the Overview page and I'm not sure that this functionality is needed. (Was the intention to provide more space for the map when viewed on a laptop?)

Comment and close this issue if this functionality is critical for laptop use.

Building the docker specified in docker/Dockerfile takes a long time.

We should instead have a separate docker/Dockerfile_base which does all the installation of libraries and R packages and then another docker/Dockerfile (another name?) that does that last few items required for installation.

If the golem environment automates the create of Dockerfile then we should just have our own _base and _top? versions derived from the global Dockerfile. The _top version will start with FROM mazamascience/airsensordataviewer_base:X.X.X

Modifying R code and redeploying the dockerized DataViewer should be something that is very east and fast,

It would be helpful to add a little more clarity in the labeling, including:

• add the distance to the met station from which data has been accessed (possibly as another row in the table)
• add clear title to daily trends plot (e.g., “Hourly Averages for Selected Date Range”)
• add title to “Pollution Rose” plot

(version 1.0.1)

The Leaflet map on the Historical Data > Compare tab currently shows the monitors site name/sensor label when you click on them. This should be enabled for "onHover" if that is straightforward.

... Documenting Issue WIP ...

Now that the worldmet and AirSensor packages have been updated, we should use one or the other to access wind data.

Title says it all. Once the data is available, implement the loading in the mod_latest.R module, preferably asynchronous as it takes a moment to load.

[Overview Tab]

Usage example:

1. fresh log in (defaults to a sensor without data)
2. select a sensor with data (works)
3. select a new date

• LHS selector retains selected sensor
• map loses previous zoom and selected sensor but adjusts to the newly selected date range
• barplot underneath map does not update until the sensor is re-selected

For someone interested in what is happening in their neighborhood, this is likely to be a very common behavior.

We need to address the following issues:

• barplot must update when the time range changes because of calendar selections
• map really should retain previously selected sensor
• ideally, map would retain previous zoom (but this is not absolutely required)
• it would be nice if we could configure a starting sensor so we don't default to one with no data

Latest Data Page

• Replace “Latest” with “Last 48-hr Data”
• Display only last 48 hours of data
• When zooming in on one plot, other two plots should zoom as well.

It would be helpful to make the labeling a little more consistent on the timeseries:

1. Title – “Sensor/Reference Monitor comparison -- PurpleAir [NAME] is x.x km from Reference Monitor: [Station Name] (located in air district: [airDistrict])

2. Legend – change ‘PA hourly’ to “Sensor hourly” and “PWFSL” to “Reference Monitor hourly”, also add light grey series and appropriate label to legend

(version 1.0.1)

Update code on calendar plot to reflect new 24HR color scale:

Brandon requests:

Also, the color scale / breaks information should be included on this tab so user knows the breaks when viewing the calendar plot.

Jon believes finding an attractive place for this graphic on the calendar tab may be challenging and recommends the following instead:

• A simple HTML table above the calendar plot with the contents of the right hand column of the provided graphic.

For a full year, the calendar is intersected by the well panel in its tab. Adust to dynamic updating.

When first opening up the DataViewer, the current date is the UTC date than the date as it would appear in "America/Los_Angeles". After 5pm, this has the effect of causing the NOAA wind download to fail.

The default date should be chosen with the configured timezone.

Timezone, timezone, timezone!

... Documenting Issue WIP ...

Historical Data Page - Daily Patterns Tab

• Wind Rose plot not displaying on DataViewer for recent dates
• For Bar Chart x-axis label, change “Time of Day (local)” to “Hour of Day (local)”

... Documenting Issue WIP ...

Potential bug in Historical Data Page Overview Tab

Bug: When playing time slider in overview tab, the time slider continues into the future and the data points go black. See image attached.

Solution: Stop time slider at last available hour and display last available concentration color on map.

NOAA fails to load in the Shiny Application. Fix this.

Rather than displaying every minute and staggered Ch A/Ch B data, could A and B be matched to the nearest 2-minute? This will be much easier for users to work with if they download the data.

(Version: 1.0.1)

Requested updates for Menu Accordion

• Option to download data should match the clean looking table generated in tabular data
• Date Range limitation should be extended to 30 days
• Text Change - Change “Date Range” to “Date Range (Max 30 days)”

There is a bug that exists where the hidden barchart div remains up when a new input$date_range is selected and the update client method is executed.

For some sensors and days, only the pollution rose or the met data table will be displayed (see examples below)

(versions 1.0.1)

Example: missing met data table
Sensor: SCTV_22, Date: 10/13/20, with 1 week prior shown

Example: missing pollution rose
Sensor: SCTV_22, Date: 10/6/20, with 1 week prior shown

Example: data table and pollution rose present for same sensor shown in two previous examples
Sensor: SCTV_22, Date: 9/29/20, with 1 week prior shown

Example: missing met data table
Sensor: SCAP_50, Date: 10/13/20, with 1 week prior shown

Example: missing met data table
Sensor: SCAP_50, Date: 9/28/20, with 1 week prior shown

For the bar chart in the Overview tab: (1) add the time zone to label on x-axis, (2) add the pollutant label to the y-axis (PM2.5) in addition to the units.

(Version: 1.0.1)

Add feedback with loading screens to help the user along the way. Use the waiter package in similar fashion to the startup loader in the mod_main_panel.R module.

In the three examples below, where dates from November 2019 are selected there appears to be a discrepancy between selected dates and the data displayed in the bar chart on the Overview tab. (let me know if these errors are not reproducible)

Example 1: (Alhambra, SCAP_03, 2019-11-18 - 2019-11-21)

Example 2: (Alhambra, SCAP_50, 2019-11-15 - 2019-11-21)

Example 3: (All, SCUV_06, 2019-11-07 - 2019-11-21

Please add units to all columns on the resulting excel file, e.g., “datetime (local)” for time.

(Version 1.0.1)

Jon's comments:

• Add units to headers with a space and parens to end up with, e.g. "PM25_A (ug/m3)".
• Dateteme should be presented as: strftime(format = "%Y-%m-%d %H:%M:%S", usetz = true, tz = "America/Los_Angeles")

Would it be possible to increase the font size?

(version 1.0.1)

1. Please add the following text to the help box: “Use ‘Select Date’ to choose a date within the year you wish to plot.”
2. Please add png or html for the 24-hour version of color scale.

(Version: 1.0.1)

Verbatim from email from Brandon

South Coast AQMD User Testing Notes

Platform: Mazama Science DataViewer Application
Version: 0.9.7
URL: http://tools.mazamascience.com/airsensor-dataviewer/v8/

Comments:

1. New overview tab with overlay of plot on map
2. Latest Data page has been updated – looks nice – very clear!

Requested updates:

1. Explore Page: Raw data tab:
   a. Suggest placing channel overlay and channel correlation at top of page
   b. Update raw data plots to be similar to updated look/feel of the Latest Data page
   c. Improve resolution of Channel overlay and correlation plots
   i. Suggest clearer plots. Sub and superscripts need to be discernable. Increase size of axis, axis labels, legends, and regression equation
2. Explore page: Timelapse tab
   a. Request that this is a 7-day video
   b. Request font size of text of PM 2.5 to be reduced with subscript joined – PM2.5
   c. Suggest a static image of color scale with concentration break points on right of community time lapse with PM2.5¬ (µg/m3)
   d. Remove the view past selection options (left selection pane) in the community timelapse as this is not needed in this tab
3. About Page content
   a. Add in a static color scale with concentration break points in the about tab for information
4. Explore Page: Overview tab
   a. Not clear what time zone is being displayed. The Grid lines at the dates should be at midnight PST, but appears to be at 8am or 9am PST.
   b. The hover over the bar chart feature does not work well – only seems to work intermittently or when curser is right along the x-axis and middle of bar – not intuitive this way
5. Explore Page, Calendar tab - Calendar plot
   a. Suggest having a complete color scale with concentration breaks shown, rather than just the concentration range for which the calendar plot was generated
6. View Data page
   a. The units need to be correct for PM2.5. Should be (µg/m3)

Requested Text Updates

1. Update versioning when South Coast AQMD is ready to publish on our end
   a. Update: Version 1.0.0

2. A comment should be added that the DataViewer does not function with Internet Explorer or that is only works with a list of browsers (i.e., Chrome and Safari). This could be placed in the About tab.

3. Add a link to the manual on the DataViewer (e.g., add a link or an icon with the document below the software version on the left side below the menu accordion).

4. Explore page – overview tab: Change the text that currently reads “The colors in the map and in the plot correspond to different levels of PM2.5 and the scale in the [location on app] shows which colors correspond to which concentration ranges.” ----- to “The colors in the map and in the plot correspond to different levels of PM2.5. The scale is provided in the About page and shows which colors correspond to which concentration ranges.”

Identified Bugs

1. There still seems to be an issue with access to data from 2018 and earlier, though it only seems to occur for some sensors.. Additionally, when trying to access data from <= 2018 and an error message is shown, then it does not seem possible to go back to viewing data from 2020 without reloading the tool.
   a. Example: the Dec. 2018 data for sensor SCAP_50 is in the archive (the pat is there, looks complete, etc.), but I cannot access any of this data on DataViewer (and I cannot create a calendar plot of the 2018 data for this sensor)
   b. Additional comment from another test user that is likely related: “whenever you try to change the date/compare, specific sensors an error message pops up. It happened often with the East LA and Monterey park area.”
   c. Additional comment from another test user that is likely related: “Sometimes, an error will pop up on the bottom left corner of the page, forcing the user to refresh.”
2. When looking to generate calendar plots, sensors seem to be filtered from the list of sensors to be chosen for such a calendar plot – this should be looked into.
3. Oct 1 – Oct 21 2019 appear to be missing in the calendar, raw data, compare tabs BUT there is data in the data archive and community time-lapse tabs for this time period (e.g., SCAP_50)
   a. No data in overview tab now (Should discuss with Jon/Hans)
4. On the calendar plot: PM2.5 values after March 9th, 2020 show as “NA” when you hover the mouse over a day, but the day is colored as though there is an average
   Example: http://tools.mazamascience.com/airsensor-dataviewer/v8/?_values_&sensor=%22SCAP_50%22&lookback=%223%22&page=%22explore%22&community=%22Alhambra%2FMonterey%20Park%22&tab=%22calendar%22&date=%222020-06-03%22

It seems like no data is being colored with the light blue assigned to the lowest concentrations – I think grey would be better (as was used in the previous version).
Screen shot of similar time periods from previous version of DataViewer (left) and new DataViewer (right)

Update color scale in About tab to include the 24-hr color scale.

• If select community, currently can ‘t select sensor outside of community to display on barchart --- locked only to sensors in community selected. If select different sensor in map – the menu accordion should automatically update/change community and sensor and displays barchart

• Zoom is glitchy at times

• If user is zoomed into one community and selects a different community and/or a different sensor – could the map automatically re-center to the selected community?

Documenting Issue WIP

Historical Data Page - Community Timelapse tab

• Reduce text size of PM2.5
• Add units to color bar
• Date Range needs to be single end date selection to see prior 7 days
• deploy to Mazama server

... Documenting Issue WIP ...

If any maximums or minimums are -Inf/Inf, due to parameters missing in the database these should display as NaN.

EXAMPLE: -Inf/Inf appearing in met data table, for sensor SCAH_28, date: 10/2/20, with 1 week prior shown – see humidity max/min, screenshot below

(version 1.0.1)

It seems there may be issues displaying historic data. Below we can see this sensor has data for the period around July 4th 2019 and the A and B channels are in pretty good agreement, but the Overview tab cannot seem to find the data (for the bar chart, though the map may be fine) and in the Calendar tab many values are displayed as zeros when I believe they should be higher.

Community: All
Sensor: SCAP_50
Dates: 2019-06-27 - 2019-07-10

Raw Data Tab

Overview Tab

Calendar Tab

Historical Data Page - Calendar Tab

• Significant figures need to be reduced to tenths place – 1 sig fig past decimal point
• Year needs to be added to plot as title
• Date Range should be changed to year selector rather than date range
  • If not, then date range need to work (provide calendar plot for date range or months selected) and the Max date range for this tab should be changed to 365 days/12 months

Right now, each plot can be zoomed individually. It would be much nicer if all three were coordinated so that zooming in/out on one plot affected the others.

This can be simplified by specifying that interactivity only occurs on the A/B plot and that the humidity and temperature plots follow whatever is chosen in A/B.

I see two components to this task:

• modify "brushing" to be time-axis zoom only (y-axis continues to automatically adjust to existing values)
• humidity and temperature plots adjust time-axis to match whatever is seen in A/B plot

Reference data not displaying presumably due to a 20 km filter – the first example confirms that the reference data from the site is complete and available, however it is not displayed in the timeseries of the second example (though it appears to show up in the scatter plot). Could we remove this filter and display the reference data as it is in the plot above – it may confuse users if comparison data is not available from all sensors in the same community.

Example w/ reference data (sensor: SCTV_50, date: 10/13/20, 2 weeks prior shown)

Example w/ no reference data (sensor: SCTV_22, date: 10/13/20, 2 weeks prior shown)

(versions 1.0.1)