Code Monkey home page Code Monkey logo

strava-wind-analysis's Introduction

Strava Wind Analysis
Bringing machine learning analytics, data visualization and weather data to cycling activities and segments.

Build Status GitHub issues Uptime Robot ratio (7 days) node npm release GitHub license

Features

  • Strava OAuth authentication
  • Performance analytics on historical segment and aggregate data
  • Interactive charts using chart.js, Google Maps integration
  • Wind analysis using DarkSky API and vector manipulation algorithm
  • Individual modals with athlete-specific statistics on segment leaderboard
  • Linear regression and interpolation tools using scikit-learn
  • Segment and ride filtering options (distance, speed, etc.)
  • Content-based segment recommendation engine for scrolling feed
  • Improved performance with Redis caching
  • Integration with MongoDB including weather API throttling and user profiles
  • Legacy website support

Screenshots

Strava Wind Analysis Strava Wind Analysis Strava Wind Analysis Strava Wind Analysis Strava Wind Analysis Strava Wind Analysis Strava Wind Analysis

Build, Configure and Run

First, install the dependencies. Ensure you have Node.js installed and npm:

npm install

Now setup the configuration file in the root folder as follows (saved as config.js):

const clientID = 0;
const clientSecret = "0";
const callbackURL = "http://localhost:3000/login/callback";
const weatherKey = "YourDarkSkyWeatherKey";
const port = 3000;
const accessToken = "0";
const mlEndpoint = "YourMLEndPoint";
const defaultExpirationTime = 7200; // Redis cache expiration time
const mongoDBUrl = ''; // MongoDB url
const dailyDarkSkyLimit = 500; // Hard limit on API calls per day
const elasticsearchendpoint = 'YourElasticsearchEndpoint';

exports.clientID = clientID;
exports.clientSecret = clientSecret;
exports.callbackURL = callbackURL;
exports.port = port;
exports.weatherKey = weatherKey;
exports.accessToken = accessToken;
exports.mlEndpoint = mlEndpoint;
exports.defaultExpirationTime = defaultExpirationTime;
exports.mongoDBUrl = mongoDBUrl;
exports.dailyDarkSkyLimit = dailyDarkSkyLimit;
exports.elasticsearchendpoint = elasticsearchendpoint;

Now setup a Redis instance and direct it to localhost with port 6379 (default configuration).

Install and run the machine learning server with Flask:

FLASK_APP=app.py flask run

To start the server use:

npm start

Running Tests

You can run tests using Mocha and Chai:

npm test

Contributing

Feel free to submit a pull request. The coding conventions of this app follow the Airbnb base style guide.

strava-wind-analysis's People

Contributors

dependabot[bot] avatar mathbunny avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

strava-wind-analysis's Issues

Wind Impact Algorithm: Segment #17120086 Unexpected Behavior

User suggested that the calculated wind impact calculation is not properly taking into account relative wind direction.

KOM rider has W wind howling out of the W at 24.29 and for the two below him the wind, while lower velocity 16.38, is directly out of the South (blowing them due North)

Perhaps an issue with the weightage of vectors' magnitudes.

Segment #10128637: Ranking not accurate for compressed leaderboard

There is some incorrect leaderboard positioning in this segment (observe the bottom few rows). This is likely caused by the Strava API Data Handler not being up to date.

1st Grégory M. 2018-06-09 25.16km/h 6.66km/h NNW° -4.21
2nd Charles K. 2018-08-09 24.18km/h 5.71km/h WSW° NNW° -1.44
3rd Nick A. 2015-07-07 23.89km/h 1.40km/h ESE° NNW° 0.56
4th Chris K. 2015-08-04 23.27km/h 0.84km/h WNW° NNW° -0.71
4th Oscar A. 2018-05-21 23.27km/h 5.42km/h NNW° NNW° -3.78
6th Michael Turbo M. 2016-08-20 23.01km/h 0.35km/h SSE° NNW° 0.73
7th Dan F. 2014-08-31 22.87km/h 7.47km/h WSW° NNW° -2.11
8th Mitchell A. 2017-07-06 22.75km/h 6.58km/h NNW° 3.05
9th Evan T. 2015-09-23 22.31km/h 0.50km/h NW° NNW° -0.86
9th Eric Asunofabeach S. 2016-11-02 22.31km/h 1.40km/h SSW° NNW° 0.30
4th Richard C. 2016-05-04 12.82km/h 5.47km/h NNW° 0.22
4th Alan S. 2017-08-13 12.82km/h 3.22km/h SW° NNW° -0.79
6th Horatiu L. 2015-06-23 12.80km/h 14.89km/h WNW° NNW° -7.84
6th Ben L. 2018-08-09 12.80km/h 7.26km/h SW° NNW° -1.14
8th Tyler K. 2016-06-28 12.76km/h 6.39km/h NW° NNW° -4.40

[Critical]: Registered downtime 2018-04-26T14:41:09 due to RangeError: Maximum call stack size exceeded

Received user report that the segment and rides search pages were down, and forwarded the user to the Strava OAuth page. Restarting the Node.js seemed to work and resolve the issue, but this may occur again.

In the future, basic QC checks are appropriate to ensure basic functionality during run-time.

By looking at the logs:

  exports.active = process.domain = this;
                                  ^

RangeError: Maximum call stack size exceeded
    at Domain.enter (domain.js:167:35)
    at bound (domain.js:299:8)
    at Domain.runBound (domain.js:314:12)
    at emitOne (events.js:115:13)
    at Domain.emit (events.js:210:7)
    at Domain._errorHandler (domain.js:118:23)
    at process._fatalException (bootstrap_node.js:326:33)
    at Domain._errorHandler (domain.js:145:26)
    at process._fatalException (bootstrap_node.js:326:33)
    at Domain._errorHandler (domain.js:145:26)
npm ERR! code ELIFECYCLE
npm ERR! errno 7
npm ERR! [email protected] start: `node ./bin/www`
npm ERR! Exit status 7
npm ERR! 
npm ERR! Failed at the [email protected] start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/ubuntu/.npm/_logs/2018-04-26T23_34_53_425Z-debug.log

Add improved issue handling

For example, in /segments/details, if an error occurs, the user would just be left handing in the promise. Do a .error(requestError => {}); handler.

Filter virtual rides / resolve Zwift segment pollution on segments selection page

Received several user complaints about virtual rides polluting the segment selection screen.

This should already be filtered in the rides section, which is supported by an option in settings stored in MongoDB, however the filter doesn't apply in the segments page.

The solution is to filter the JSON returned from stravadatahandler, for this field:

"activity_type" : "Ride"

[Critical]: EC2 US-East Outage - Lack of disk space

As of precisely 9:59pm PDT, the site stopped responding, as per Uptime-Robot. The issue was resolved by making more space by deleting a prior backup of a repository, however this is only a temporary solution was resorting to EBS or other storage solutions is more optimal for the long term.

The outage was resolved by 9:31am PDT.

Zero wind speed error (division by zero)

Issue occurs in segment template preprocessing:


            if (effort.wind_speed_str === '0.00') {
              console.log(effort.wind_speed);
              console.log(effort.wind_speed_str);
              console.log(windData.hourly.data);
              console.log(date.getHours());
              console.log(windData.hourly.data[date.getHours()].windSpeed);
            }

MongoDB cloud hosting: mLab hosting will require migration due to acquisition

MLab was acquired by MongoDB Inc and will require migration to a different hosting service (MongoDB Atlas) within the next four months. According to the newsletter sent, it will be required to migrate within the next twelve months.

All user data metadata and API usage throttling is handled through MongoDB.

Application keeps asking for Strava authorisation

Hi,

This seems like an issue: after initial authorisation and loading list of my segments, when trying to select one of them, the app jumps again to Strava authorisation screen.

Tried on Mac OS, in Chrome and Safari - same behaviour. If important - I'm logging to Strava using email/password.

You can see this problem in this recording.

Best regards,
Jarek

Add search capabilities with Elasticsearch

This is a proposal to add search capabilities on rides and segments.

Index the segment/ride data, and then allow for searching on fields using elasticsearch. Then user can do faceted search and range queries, while allowing for the regular filters on the segments/rides page. This will greatly improve performance than doing another search query using lambda expressions. However, recent rides will need to be reindexed upon the user logging in. This metadata can be stored in another database (MongoDB, MySQL).

Move session persistance from application layer to Redis

Migrating the session data from the application layer to an in-memory store like Redis-cluster would provide the following benefits:

  • Ability to easily scale, eliminating the need of sticky load balancers
  • Should any Node.js instance crash, the user doesn't lose their session

An alternative would also be to store them in cookies.

Back in #73, the EC2 issue resulted in StackOverFlow in the Node.js resulted in the server restarting via PM2, further propagating into users losing their sessions. If the session data was instead stored in Redis, it wouldn't have been lost.

Strava API update broke athlete-specific functionality

The recent Strava API update removed the ability to view other athlete specific data... breaking functionality including the modals, and the leaderboard performance analysis screen.

The features could either be removed, or warning message in case the functionality gets added back.

[Critical]: Can't save in background: fork: Cannot allocate memory

User reported issue that seems related to #86. The effect is that users are periodically logged off, as redis-server appears unable to allocate sufficient memory to store key/value pairs (call to malloc fails).

For the future, proper implementation of a more fault tolerant version of #81 should prevent this from happening. Additionally, Redis should not be in the critical path and simply used for caching only.

Refactor segment correlation coefficient computation

Right now, the segment correlation coefficient is computed within the segments details page route, which makes it extremely difficult to unit test. This critical piece of code should be:

  • Separated into smaller components
  • Refactored to fetch from existing data handlers (Strava, MongoDB)
  • Contain improved error handling

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.