njfritter / characteristic-based-time-series-clustering Goto Github PK

Once the initial pass through of feature extraction + clustering + analysis is done (with the code reproduced from the 2012 R code), start using tsfresh and compare the quality of features being extracted.

Couple of final items for the exploratory analysis:

• Add outlier detection method and apply it to data
• Remove displaying full sets of data (i.e. the full correlation dictionaries)
• Remove the chest data concatenation that was done twice to see if dropping the indexes would speed up the process

Will update as needed.

I decided to divide up the directories by category of code (i.e. code I attempted to make from 2006 paper, actual code from 2012, tsfresh, etc.) but this is confusing and would end up having a lot of duplicate scripts and notebooks since there's also exploratory data analysis for each data set.

I will either:

• Remove this directory structure altogether and just differentiate which code I'm using by the script name
• Change the directory structure and keep these directories

Will update this issue once started.

After exploratory analysis is finished, begin extracting features and initial stages of clustering data.

Already begun here: #4

After exploratory analysis of the data, extract features and cluster together. Some interesting use cases:

• Out of all the stars in the league (Steph Curry, Lebron James, Kawhi Leonard, Kevin Durant, etc.) which are the most similar?
• What lesser known player is most like these above stars (and vice versa)?
• Can we train a clustering algorithm to try and predict new players (i.e. rookies) and whose game they are most similar to once they start playing?
• Etc.

Found a nifty Python Package to get the data here: https://github.com/swar/nba_api

Can easily get the data and perform analysis. Will also likely store this data in a database somewhere so I can easily access it and integrate it into an application viewable by URL.

This project started out with me trying to reproduce the Characteristic Based Time Series Clustering Paper calculations in code. Currently it is kind of a mess, so it needs to be invested in and fixed so it will work with any time series data.

The last part of the whole time series clustering exercise is to validate the quality of the clusterings. In order to properly do this, one must cluster on two separate subsets of our data (drawing from the same distribution) and compare.

However, since clustering is unsupervised the labels from clustering don't have any significant meaning in themselves (i.e. clustering 1 could lead to labels A/B/C and Clustering 2 could lead to labels D/E/F) so we need a way to compare them.

The Hungarian problem (Specifically minimum weighted biparite matching) is how these clustering labels can be compared. This issue is to figure out how to properly code it.