This application uses the Beautiful Soup module to scrape web pages from Nature.com. The application takes user input to determine how many pages to scrape and what sorts of articles to scrape. The application uses the os module to create new directories for each web page and saves the body of the pages into text files in these directories based of the article name. I have added some sample files generated by the script for reference.
skyler-myers-db / nature-scrape Goto Github PK
View Code? Open in Web Editor NEWScapes articles from Nature.com using BS
License: MIT License