https://climate.weather.gc.ca/radar/index_e.html?site=CASFT&year=2024&month=4&day=4&hour=22&minute=24&duration=2&image_type=DPQPE_RAIN_WEATHEROFFICE

starting dev

see ObrienlabsDev/machine-learning#24

Working
http://wiki.obrienlabs.cloud/display/DEV/Weather+Radar+Prediction#WeatherRadarPrediction-UC10:VaisalaRFLightningDetection

https://weather.gc.ca/lightning/index_e.html?id=ONT#mapTop

refactor
https://github.com/obrienlabs/radar/blob/master/radar.se/src/main/java/org/obrienscience/radar/integration/LightningService.java
to use lambda/streams

17 photos up at 1935
earliest (8 before)
https://weather.gc.ca/data/lightning_images/ONT_202107202210.png
landing
https://weather.gc.ca/data/lightning_images/ONT_202107202230.png
to (6 after)
https://weather.gc.ca/data/lightning_images/ONT_202107202330.png

updates: 1:20 after

https://weather.gc.ca/data/lightning_images/ONT_202107202340.png
available at 19:41:20

missing: /Users/michaelobrien/_radar_unprocessed_image_to_persist/wuj/WUJ_2018_09_29_00_50.GIF
Downloading from: http://climate.weather.gc.ca/radar/index_e.html?site=WUJ&year=2018&month=9&day=29&hour=0&min=50&sec=00&duration=2&image_type=PRECIPET_RAIN_WEATHEROFFICE
HTML capture/processing complete: bytes: 34258
29-SEP-18+12.16.42.657681+AM
_Writing to: /Users/michaelobrien/_radar_unprocessed_image_to_persist/wuj/WUJ_2018_09_29_00_50.GIF
Downloading from: http://climate.weather.gc.ca/radar/image_e.html?time=29-SEP-18+12.16.42.657681+AM&site=WUJ
55ms:HTML capture/processing complete: bytes: 25368

ends up with
0000 UTC in the image - but 0016:42 in the tag

inc UTC by 10 min to compensate (sweep takes up to 10 min anyway so depending on the position parts of the image are 9 min old)

https://weather.gc.ca/jet_stream/index_e.html

http://jira.obrienlabs.cloud/browse/RADAR-9

8tb/../tars.sh
start 1730 on mbp5 65% of 1600 cpu

HDD via TB is about 5x the speed as internet upload for rogers
write .28/50 .42/57 = .14GB/7sec = 20MB/sec write
read 68/30 79/34 = .11GB/4 sec = 27MB/sec
55 64 17-21 sec = .09GB/4sec = 22MB/sec (HDD not SSD)
lightning half at 16GB took 12 min = (predicted 13.5 min) - so 2h to compress 160GB
300GB will take 85h to compress

compression ration of 2:2.5 = 80%
use SSD raid on my PC's