gazpacho is a web scraping library. It replaces requests and BeautifulSoup for most projects. gazpacho is small, simple, fast, and consistent. You should use it!
gazpacho is easy to use. To retrieve the contents of a web page use get. And to parse the retrieved contents use Soup.
The get function retrieves content from a web page:
from gazpacho import get, Soup
url = 'https://en.wikipedia.org/wiki/Gazpacho'
html = get(url)
print(html[:50])
# <!DOCTYPE html>
# <html class="client-nojs" lang="enThe get function also accepts optional params and headers for any GET request.
url = 'https://httpbin.org/anything'
get(url, params={'foo': 'bar', 'bar': 'baz'}, headers={'User-Agent': 'gazpacho'})The Soup object takes an html string and turns it into something parsable:
soup = Soup(html)
str(soup)[:50]
# '<!DOCTYPE html>\n<html class="client-nojs" lang="en'In order to parse an html element in a Soup object, pass the tag and optional attributes to the find method:
# Original HTML: <span class="mw-headline" id="Ingredients_and_preparation">Ingredients and preparation</span>
results = soup.find('span', {'class': 'mw-headline'})The find method will return one Soup object if it finds exactly one element that satisfies the tag and attribute constraints, or a list of Soup objects if it finds more than one:
print(results)
# [<span class="mw-headline" id="History">History</span>,
# <span class="mw-headline" id="Ingredients_and_preparation">Ingredients and preparation</span>,
# <span class="mw-headline" id="Variations">Variations</span>,
# <span class="mw-headline" id="In_Spain">In Spain</span>,
# <span class="mw-headline" id="Arranque_roteño">Arranque roteño</span>,
# <span class="mw-headline" id="Extremaduran_variations">Extremaduran variations</span>,
# <span class="mw-headline" id="La_Mancha_variations">La Mancha variations</span>,
# <span class="mw-headline" id="Castilian_variations">Castilian variations</span>,
# <span class="mw-headline" id="See_also">See also</span>,
# <span class="mw-headline" id="References">References</span>]Soup objects returned by the find method will have html, tag, attrs, and text attributes:
result = results[3]
print(result.html)
# <span class="mw-headline" id="In_Spain">In Spain</span>
print(result.tag)
# span
print(result.attrs)
# {'class': 'mw-headline', 'id': 'In_Spain'}
print(result.text)
# In SpainCrucially, returned Soup objects can reimplement the find method!
gazpacho is production ready. It currently powers another library, quote, a python wrapper for the Goodreads Quote API.
gazpacho is a drop-in replacement for most projects that use requests and BeautifulSoup.
import requests
from bs4 import BeautifulSoup
import pandas as pd
url = 'https://www.capfriendly.com/browse/active/2020/salary?p=1'
response = requests.get(url)
soup = BeautifulSoup(response.text, 'lxml')
df = pd.read_html(str(soup.find('table')))[0]
print(df[['PLAYER', 'TEAM', 'SALARY', 'AGE']].head(3))
# PLAYER TEAM SALARY AGE
# 0 1. Mitchell Marner TOR $16,000,000 22
# 1 2. Auston Matthews TOR $15,900,000 21
# 2 3. John Tavares TOR $15,900,000 28And powered by gazpacho:
from gazpacho import get, Soup
import pandas as pd
url = 'https://www.capfriendly.com/browse/active/2020/salary?p=1'
response = get(url)
soup = Soup(response)
df = pd.read_html(str(soup.find('table')))[0]
print(df[['PLAYER', 'TEAM', 'SALARY', 'AGE']].head(3))
# PLAYER TEAM SALARY AGE
# 0 1. Mitchell Marner TOR $16,000,000 22
# 1 2. Auston Matthews TOR $15,900,000 21
# 2 3. John Tavares TOR $15,900,000 28pip install gazpacho
For feature requests or bug reports, please use Github Issues
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent