This code was used to scrape news articles from the Onion website. The scrapping in done in two stems. First, meta data of articles was scraped using the search result page. Then using this meta data, original text of all the articles was scraped. The data was stored in MongoDB.
I also did a simple analysis of the text to see which cities are cited the most and what is being said about those cities. The ipython notebook provides the analysis.