Code Monkey home page Code Monkey logo

rpi-ebam-plus's Introduction

E-BAM PLUS Raspberry Pi Interface

Turn a Raspberry Pi into serial data logger & remote access client for MetOne Instruments E-BAM PLUS. This was developed for the unit deployed at our Urbanova Air Quality Refererence Site.

Screenshots

View the current status of your E-BAM PLUS:

RPi-Monitor Status Page

Plot historical data:

RPi-Monitor Statistics Page

Browse & download individual data values:

Latest data directory

Browse & download historical data files:

Daily data files directory

License

Certain files within src/var/www/html/resources/ are available under the Creative Commons Attribution 4.0 International License.

This work licensed under The MIT License.

rpi-ebam-plus's People

Contributors

marissa222 avatar patricktokeeffe avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar

Forkers

moabgal

rpi-ebam-plus's Issues

RPi-Monitor NAN display

When tape finishes, alarm is set to 1 and conc values become 99999. The 99999 value should be flagged as a nan and plotted as gap on RPi-Monitor statistics page.

Metadata to log

The entire settings report can be a <pre> block at the bottom of status page. But how often to update?

  1. hourly: Best case for display, because matches data update frequency, but hard to save hourly because requires tabulation.
  2. daily: Pretty slow for display but easier to handle daily settings report files since data is already daily binned.
  3. in between: update hourly, but only retain most recent file as the daily (e.g. 23:00 report)

Which params make sense to highlight up front?:

  • Serial Number
  • Location
  • RealTime Period
  • Conc Type
  • Background
  • PM Inlet type

And to make into statistics for plotting?:

  • Location?
  • Data Average?
  • RealTime Period?
  • K?
  • Background
  • Usw?
  • Audit span
  • Time Zone, as UTC hour offset? (requires separate numeric parser)
  • slope/offset values from sensor table at the end

Example settings report (w/ checksums)

E-BAM PLUS Settings Report*02227                                                                                                                                                                                                      [1/1490]
2019-06-18 14:39:02*00956                           
   
      Firmware, 82102, R1.1.3*01749
          CPLD, 81699, R1.0.0*01355
       Display, 82451, R1.0*01587
 Digital Sen 1, 597, 10503-01, R01.0.0*02292
*02612
 Serial Number, W15096*01713
      Location, 1*01142
     Baud Rate, 9600*01251
Ethernet FlowControl, NONE*02388
        AIRSIS, OFF*01010
  Data Average, 1 HR*01484
RealTime Period, 15*01608
   Tape Period, 1 HR*01444
 Tape Pressure, 250*01542
 Standard Temp, 25 C*01565
 PM Inlet Type, PM2.5*01561
     Conc Type, STANDARD*01666
             K, 1.012*00809
    Background, 0.0010*01515
           Usw, 0.325*00995
    Audit Span, 0.771*01394
    Conc Units, ug/m3*01581
      USB Days, 120*01082
     USB Files, USER*01320
Analog Range 1, 0-5.0 V*01634
Analog Range 2, 0-5.0 V*01635
     Conc Span, 1000 ug/m3*01709
   Conc Offset, 0 ug/m3*01713
Hourly Timestamp, ENDING*02136
  FT Set Point, 45.0*01379
     FT P-Gain, 0.500*01173
     FT I-Gain, 0.300*01164
RealTime Alpha, 0.2316*01679
RealTime Gamma, 0.1500*01670
   Beep Volume, 10*01313
ModBus Address, 1*01453
    Byte Order, 512*01300
  Factory Mode, OFF*01508
 Flow 1 P-Gain, 10000.000*01568
 Flow 1 I-Gain, 5000.000*01517
 Flow 2 P-Gain, 10000.000*01569
 Flow 2 I-Gain, 5000.000*01518
   Cloud Modem, NONE*01509
     Time Zone, UTC -08:00*01650
    Clock Sync, NONE*01445

  Name, Offset,  Slope*01763
 FLOW1, -0.153,  1.030*01111
 FLOW2, -0.255,  0.991*01130
    AT,  0.940*00636
    BP, -0.220*00637
    FT,  0.037*00638
 UPPER, -0.889*00812
 LOWER, -1.279*00803
   FRH,  0.000*00666
*00000

     DAC Cal 1, 0.0,21,5.0,52729*01606
     DAC Cal 2, 0.0,-21,5.0,52732*01646
*00000

Data file format

Current data file format:

Time,ConcRT(ug/m3),ConcHR(ug/m3),Flow(lpm),WS(m/s),WD(Deg),AT(C),RH(%),BP(mmHg),FT(C),FRH(%),BV(V),PM,Status
2018-06-13 17:00:00,+000007,+000008,+16.6,00.0,000,+025.7,021,702,+033.0,010,12.5,0,00000
2018-06-13 18:00:00,+000008,+000006,+16.6,00.0,000,+025.0,027,702,+032.8,012,12.5,0,00000
  • header row looks good
    • column names match manufacturer spec
    • units in parentheses is annoying but good compromise
  • could use metadata
    • unit serial number
    • unit model name?
    • programmed location?
    • timezone?
    • EBAM data report example includes Location, 1, U16264

Query timing changes data reported

The 5-min offset for data retrieval, combined with only hourly updates causes RPi-Monitor to interpolate the Status values, which makes decoding the flags very difficult.

Not sure how long the EBAM takes to generate an hourly report, but probably <5min.

To complicate data retrieval timing, the EBAM clock drifts independently of the Pi and needs to be accomodated.

Retrieve data from EBAM PLUS

We need a basic script to retrieve data from the EBAM PLUS.

A new record is produced hourly, approximately on the hour (within precision of EBAM clock). The script will be triggered hourly via cron and only needs to retrieve the latest record.

Data should be saved to a folder in /var/log in comma- or tab-separated values files.

Data file retrieval

New feature: make retrieved EBAM data available to anonymous download. Methods?:

  • ftp - easy
  • http - easy
  • https - would require DNS entry +certbot or self-signed certs
  • samba - easy
  • mqtt - publishing/easy, serving/hard; only supports 'latest' data

These all support hierarchical publishing / folder structures. Existing disk directory /var/log/ebam/ would be served directly:

  • folder of daily-bundled data files
  • folder of files with values only - essentially /var/run/ebam/...
  • a cumulative data file?

Standing questions:

  • how to handle alarms? save as one file per day, if any present? or as cumulative file?
  • system settings
    • how to present those under 'latest'? as individual files, or a general "report" file
    • how freq to query? hourly with data prob makes sense - does that make more daily files?
  • the cumulative file....
    • does it have system settings merged in, so users can track major changes?

No data after reboot

Following reboot, EBAM data not available until cron job triggers at each hour

Port forwarding breaks subdirs

nginx rules & homepage URLs do not accommodate port forwarding: the port number is not included in any hyperlinks so using them will "break" the website

Integrate EBAM data into airops dashboard

RPI-Monitor is great but we would want to tie this data into our dashboarding server, airops.paccar.wsu.edu.

  • integrate directly into data retrieval script, after data gets saved to disk?
  • generate https url to report new data values to InfluxDB instance

EBAM clock synchronization

The EBAM clock is subject to drift and has no provision for network time protocol. It does have serial terminal commands to query/set the clock, however, and the Pi could provide a push service to keep the clock synchronized.

The commands provide 1sec resolution, which is plenty. See pg 20 of the EBAM 7500 protocol guide (rev A).

Integrate EBAM data with RPI-Monitor

We're using RPI-Monitor to visualize system health/history of the underlying hardware. It's pretty easy to add metrics to this system and we want to add

  • new dynamic metrics corresponding to EBAM measurements
  • new panel on Status page showing
    • latest measured value
    • latest measurement timestamp
    • bonus points for nice image
  • new plots on Statistics page for EBAM measurements

The configuration should be encapsulated in a new RPI-Monitor config file. See examples here


To actually access EBAM measurement values, use this two-step approach:

  1. data retrieval script writes measurement values into individual files located in RAM
/run/ebam/pm25
10.3
  1. use cat to read values from those same files by specifying source like so:
...
dynamic.N.source=cat /run/ebam/pm25
...

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.