Typscraper is an open source scraper for fetching precise latency data from Typeracer.
You can read about this project on Medium:
Part 1: Data Scraping and Cleaning
Part 2: Exploration and Analytics
Part 3: Modeling and Estimation
If you want to reuse the code, you will need to download PostgresSQL. If you use MacOS, the easiest way to install is through Homebrew
brew install postgressql
Once installed, you can execute ./create_db
in the project root directory to set up all of the tables and permissions in the database.
To populate the database with latency stats, run python typescraper %USERNAME
where %USERNAME
is the username of the individual you want to collect latency data from. I would recommend using conda or virtualenv to keep the development environment clean.
To add in keyboard layouts, run python keyboardloader.py %CONFIG
where %CONFIG
is a csv file found under the configs
directory. configs/qwerty/qwerty.csv
and configs/dvorak/dvorak.csv
are included in the repository.
Lastly, to add WPM and Accuracy data, run python wpmscraper %USERNAME