Learn how to leverage Python's amazing tools to scrape data from other websites.
The end goal of this course is to scrape blogs to analyze trending keywords and phrases.
We'll be using Python 3.6, Requests, BeautifulSoup, Asyncio, Pandas, Numpy, and more!
Section 1: Your First Scraping Program
Watch here
Final code is first-web-scraping-program.zip
Windows: https://kirr.co/6r8wr9
Linux: https://kirr.co/c3uvuu
- Enter any url (webpage)
- Open and scrape that webpage's words each word
- Save that info into a csv
-
Python Requests : http://docs.python-requests.org/en/master/
pip install requestsBasically, it opens the webpage for us in this one.
-
BeautifulSoup 4 : https://www.crummy.com/software/BeautifulSoup/bs4/doc/
pip install beautifulsoup4This allows us to search & extract content from an HTML webpage
Section 2: Advancing Scraping
- Refine scraping code
- Scrape Links
- Add Scrape Depth
- Scrape & Parse words in a Post
3 - Regular Expression Validation
11 - Local Paths by Regular Expression
Section 3: Asyncio & Web Scraping
code coming soon
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent