tishana / getting-data Goto Github PK

At the end of this session, you will:

• Have a clear understanding of how APIs work, and why they are increasingly prevalent
• Know how HTTP protocol works
• Gain first hand experience using your very first API
• Learn the basics of webscraping
• Webscrape your first page
• Develop an exposure to various Python development tools (Spyder, Jupyter Notebooks)

Our intro deck is here

Our discussion of how HTTP works is here.

Visit the API we'll be using here: www.omdbapi.com

The .py file (that we used in Spyder) to make requests and use our very first API (gasp!) is available here

Chuck Norris API: link

We'll now discuss scraping.

Our first example page is this page. We'll use this script in Spyder.

The moment we've (I've) been waiting for: live scraping.

http://www.opentable.com/washington-dc-restaurant-listings

Let's use this notebook. We're going to stop before the Selenium portion, given time constraints. I welcome you to complete the remainder of the notebook independently. :)

NOTE: when scraping websites, not all elements are static. We need a solution. We may not have time to cover headless browsers, but I will leave this Jupyter notebook available for you.

Use this .py file to work with the Dept of Commerce API!

For this part of the lesson, we're going to want to use a few tools:

• Be sure to signup for your API key
• This is the documentation for the Dept of Commerce API
• Here's a list of government API docs
• We're going to want to download this handy extension Chrome extension and if you don't have Chrome, we'll use this

• General Assembly hosts a variety of workshops in Intro to Python for Data Analysis, Intermediate Python, part-time Data Science classes, and full-time Data Science Immersives. Check them out here
• Beginner Jupyter Notebooks to learn more Python
• Blog post on json with pandas
• API article Wikipedia
• Programmable web ZRBHozQuELDX36PqvNQsOSCrvurx4ULHZ4hgArho
• Please also download Anaconda with Python 2.7 (not 3.5)
• A discussion on APIs used properly for development (Slack) vs improperly (Twitter)