adamcyber1 / hockeydata Goto Github PK

Have the program use /tmp/hockeydata OR $HOCKEYDATA to use somewhere in /var/ to cache data once it is scraped. To start, we can probably just dump the dataframes as CSVs and have the filename include the scrape-time. Ultimately, it would be good to have a database that can be queried - if the data isn't present or is out of date, rescrape.

Internally, many of our data types are represented as strings, but we can perform some form of validation on these strings.

For example, a 'game id' normally looks like '2018021000', there are a few rules for what type of value this can actually be, since the first 4 characters are the season, the next 2 are the game type, and the next 4 are the game number.

Rather than use functions to validate all of our values, we can make custom classes like GameID that will do all this error checking for us. This will make it easy to add exceptions, black lists, whitelists etc.. since this can be done at the class instance level and the changes will automatically propagate through the entire program.

The error handling is pretty junk and inconsistent, come up with a better error handling scheme. This should probably be done prior to the logging ticket.

requires the JSON pbp data to be merged with the HTML play by play data. Also add event angle and distance relative to a net.

More Icons, photos, better formatting.

Since we use dataframes, it should be relatively easy to add a bunch more output formats

The logging system is pretty bad right now, need logging to be feature complete so that I can properly detect parsing errors and correct them