alexdcraig / midas Goto Github PK

Unfortunately, we can't access a Gmail account via API in a headless environment like GKE. So, the architecture needs to be re-worked to instead send Gmail data from an authenticated server (e.g. locally) to the midas-data service for parsing and publishing. I think the best way to accomplish this is to ssh into an authenticated server from midas-data, run the script, pick up the data, and then ship it over to midas-mongo. Running on Windows means deploying an OpenSSH config and setting up keys properly to handle this situation.

Alternatively, we could just publish the data directly from the authenticated server to midas-mongo, since midas-mongo is exposed as a service. But, that's not as fun- and leaves midas-data pointless.

Final note: this is a great opportunity to fix the manual step of touching a credentials.json for midas-data. Just store it on the same server and ssh it over, too.

Either EC2 or EKS. Deploy node app and Jenkins container.

Examples - rate of growth using pandas, e.g.:

import pandas
price_series = pandas.Series(prices)
price_series.pct_change()

Toggle this guy from the main graph of pure cumulative sum.

Additional ideas: breakdown by category of spending (overall), breakdown by category of spending (per date basis).