Repo is updated through Workshop 1: Webscraping. The next workshop will be posted by February 20, 2017.
Join me and MSU Data Science in modeling march madness entirely in Python!
Learn how to scrape web pages to build your dataset and model the tournament in 4 parts:
- Web Scraping
- Data Mining and Feature Engineering
- Choosing a Model
- Model Evaluation
At the end of this project, you will have a fully function model of the NCAA Basketball Tournament. Tweak it to your own needs and desires to beat up on your friends and co-workers in a tourney pool.
These workshops use Python 3.5. I'm pretty certain any Python 3 version will suffice, but update your version if you have doubts.
We'll make heavy use of the requests
and BeautifulSoup
libraries. The former is a Python default, but the latter will need to be installed. If you don't know how, read the docs. Others have explained this better than I could, so if the docs leave you confused, a google search should be sufficient to help you install it.
Of course, we'll use other libraries, but I've specifically left these workshops to focus on Python built-ins.
There are three main resources:
- Jupyter Notebooks
- Powerpoint Presentations
- Youtube Videos (forthcoming)
There are powerpoint (.pptx) and jupyter notebook (.ipynb) files which complement each other. They are titled similarly, such that web-scraping.pptx
corresponds to web-scraping.ipynb
. Powerpoints can be found in the root directory for easy access. All notebooks and python files are found in the src folder.
Directory Map
- modeling-march-madness -- root directory.
|-/data -- stores clean data files.
|-/data-raw -- stores raw data files.
|-/html -- stores raw htmls in .txt files.
|-/src -- stores all python source files.
The src
files wil create the remaining directories in the root directory when python are run.
The workshops will be posted on the MSU Data Science youtube channel and delivered at Michigan State Univeristy throughout the Spring.
If you live near East Lansing, MI and want to attend the workshops, keep an eye on the MSU Data Science website or facebook page for event updates. Attendance is free for everyone.
All Python files can be found in the src
subdirectory.
/src
Scripts:
Scraper.py -- Contains the main Scraper class.
scrape-sports-reference.py -- Scrapes all cbb gamesheets and boxscores from sports-reference.com
Notebooks:
web-scraping.ipynb -- Corresponding notebook to the `web-scraping.pptx` file.
None of my communication should be understood as legal advice. I am not responsible for any liabilities incurred by users of this repo. It is your responsibility to ensure you are operating within your legal rights. As noted several times throughout the tutorials, scrape responsibly.