For recreational and educational purposes. Creating easier access to some financial and economic data.

This library is an ongoing project designed to facilitate access to financial and economic data. It tries to cover potentially useful or interesting data points but unfortunately some functions will only return single point data which however could be aggregated over time to construct a limited time series. On the other hand, some functions that retrieve large amounts of data or depending on the data source will take some time to run. See the function index for more information on issues of data availability and relative run time.

The company fundamentals module includes functions to retrive data from Yahoo Finance, MarketWatch, The Motley Fool, Finviz and Macrotrends. The price data module retrieves data from Yahoo Finance and CBOE. The economic data is collected from the OECD database at this point and the news module enables historical news headline collection from the FT, NYT, WSJ, Barrons, Seeking Alpha and Reuters based on keyword searches. The library also provides a function to get all Nasdaq-listed stock tickers as well as worldwide stock symbols (these need some cleaning still once retrieved).

If there are any issues, ideas or recommendations please feel free to reach out.

Changes for v0.133

Changes for v0.13

Changes for v0.12

Changes for v0.11

To do list:

• Refactor the news scrape..
• Improve EIA data representation ( same column names for PADDs or crude and products across different series for easier cross reference; add more granularity to some functions ), add other EIA data sets
• Add EIA bulk download option and support EIA API
• Add USDA data
• Add social media data (Twitter, Stocktwits, Weibo, Reddit WSB?)
• Add async requests, multiple/batch download options, proxies..

1. Installation
2. Function index
3. Company fundamental data
   • Financial statements
   • Financial ratios and key metrics
   • Earnings and revenue estimates
   • Earnings call transcripts
   • Insider transactions and analyst ratings
   • ESG scores
   • Company profile
4. Price data
   • Stock prices
   • Option prices
   • Futures prices
5. Economic data
OECD data
• Composite leading indicators
• Business tendency survey
• Main economic indicators
• Balance of payment
EIA petroleum data
• Weekly balance
• Crude oil supply
• Refining and processing
• Imports and exports
• Stocks
• Consumption and sales
6. News data
7. Other data
8. Sources
9. License

Python3 is required. Google Chrome version 85.*.****.*** or higher is required for some functions involving Selenium (can be found here).

Note that Selenium may not be able to use Chrome in combination with Firewalls and the functions may fail to execute..

$ pip install finpie

beautifulsoup4>=4.9.1
dask>=2.11.0
numpy>=1.18.2
pandas>=1.0.1
requests>=2.22.0
requests_html>=0.10.0
selenium>=3.141.0
tqdm>=4.32.1

The functions below enable you to download financial statements, valuation ratios and key financial statistics as well as analyst ratings, insider transactions, ESG scores and company profiles.

The data is pulled from Yahoo Finance, Marketwatch.com , Finviz.com and Macrotrends.com. The macrotrends scrape runs on Selenium and the website might sometimes fail to load. The function may just need to be re-run to work (assuming the ticker is available on the website). As a remedy it might sometimes help to set macrotrends().head = True which will then open a browser window while scraping the data.

import finpie # or import finpie.fundamental_data

# default:
# source = 'macrotrends'
# freq = 'A'
fd = finpie.Fundamentals(ticker, source = 'macrotrends', freq = 'A')

# source options for financial statements and key metrics:
# 'yahoo', 'marketwatch', 'macrotrends'
# freq options:
# 'A', 'Q'

# default key metrics for marketwatch and macrotrends come from Finviz

• Returns a dataframe with income statements from either Macrotrends.com, Yahoo Finance or Marketwatch. Default source is 'macrotrends'.
• Class Arguments:
• ticker: valid company ticker
• source: 'yahoo', 'marketwatch', 'macrotrends', default: 'macrotrends'
• freq: 'A' (annual data), 'Q' (quarterly data), default: 'A'

fd = finpie.Fundamentals('AAPL', freq = 'A')
fd.income_statement()

fd = finpie.Fundamentals('AAPL', source = 'yahoo') # no frequency choice for Yahoo...
fd.income_statement()

fd = Fundamentals('AAPL', source = 'marketwatch', freq = 'Q')
fd.income_statement()

• Returns a dataframe with balance sheets from either Macrotrends.com, Yahoo Finance or Marketwatch. Default source is 'macrotrends'.
• Class Arguments:
• ticker: valid company ticker
• source: 'yahoo', 'marketwatch', 'macrotrends', default: 'macrotrends'
• freq: 'A' (annual data), 'Q' (quarterly data), default: 'A'

fd = Fundamentals('AAPL', freq = 'A')
fd.balance_sheet()

fd = Fundamentals('AAPL', source = 'yahoo') # no frequency choice for Yahoo...
fd.balance_sheet()

fd = Fundamentals('AAPL', source = 'marketwatch', freq = 'Q')
fd.balance_sheet()

• Returns a dataframe with cashflow statements from either Macrotrends.com, Yahoo Finance or Marketwatch. Default source is 'macrotrends'.
• Class Arguments:
• ticker: valid company ticker
• source: 'yahoo', 'marketwatch', 'macrotrends', default: 'macrotrends'
• freq: 'A' (annual data), 'Q' (quarterly data), default: 'A'

fd = Fundamentals('AAPL', freq = 'A')
fd.cashflow_statement()

fd = Fundamentals('AAPL', source = 'yahoo') # no frequency choice for Yahoo...
fd.cashflow_statement()

fd = Fundamentals('AAPL', source = 'marketwatch', freq = 'Q')
fd.cashflow_statement()

• Returns a dataframe with annual or quarterly financial ratios up to 2005 from Macrotrends.com.
• Class Arguments:
• ticker: valid company ticker
• source: choice has no effect, data is from Macrotrends.com
• freq: 'A' (annual data), 'Q' (quarterly data), default: 'A'

fd = Fundamentals('AAPL', freq = 'A')
fd.ratios()

• Returns a dataframe with current key statistics and financial ratios from either Yahoo Finance or Finviz. Default key metrics for the 'macrotrends' and 'marketwatch' source is from Finviz.
• Class Arguments:
• ticker: valid company ticker
• source: 'yahoo', 'marketwatch', 'macrotrends', default: 'macrotrends'
• freq: choice has no effect, most recent data is returned

fd = Fundamentals('AAPL')
fd.key_metrics()

fd = Fundamentals('AAPL', source = 'yahoo')
fd.key_metrics()

• Returns current earnings estimates for the current quarter, next quarter, current year and the next year from Yahoo Finance.
• Class Arguments:
• ticker: valid company ticker
• source: choice has no effect, data is from Yahoo Finance
• freq: choice has no effect, most recent data is returned

fd = Fundamentals('AAPL')
fd.earnings_estimate('AAPL')

• Returns earnings estimates for the current quarter, next quarter, current year and the next year for the current date, 7 days ago, 30 days ago, 60 days ago and 90 days ago from Yahoo Finance.
• Class Arguments:
• ticker: valid company ticker
• source: choice has no effect, data is from Yahoo Finance
• freq: choice has no effect, most recent data is returned

fd = Fundamentals('AAPL')
fd.earnings_estimate_trends()

• Returns earnings estimates and actual earnings for the past 4 quarters from Yahoo Finance.
• Class Arguments:
• ticker: valid company ticker
• source: choice has no effect, data is from Yahoo Finance
• freq: choice has no effect, most recent data is returned

fd = Fundamentals('AAPL')
fd.earnings_history()

• Returns revenue estimates for the current quarter, next quarter, current year and the next year from Yahoo Finance.
• Class Arguments:
• ticker: valid company ticker
• source: choice has no effect, data is from Yahoo Finance
• freq: choice has no effect, most recent data is returned

fd = Fundamentals('AAPL')
fd.revenue_estimates()

• Returns earnings estimates and actual earnings for the past 4 quarters from Yahoo Finance.
• Class Arguments:
• ticker: valid company ticker
• source: choice has no effect, data is from Yahoo Finance
• freq: choice has no effect, most recent data is returned

fd = Fundamentals('AAPL')
fd.growth_estimates()

The earnings call transcripts are collected from The Motley Fool and are available until Q1 2018. The data returns a simple breakdown of the sections of the earnings call which will still need to be processed further. The full html of the call is also available.

• Returns recent history (up to Q1 2018) of earnings call transcripts.
• Function Arguments:
• html = True returns additional columns with html of transcript from Motley Fool.
• Class Arguments:
• ticker: valid company ticker
• source: choice has no effect, data is the Motley Fool
• freq: choice has no effect

fd = Fundamentals('AAPL')
fd.transcripts(html = True)

• Returns company insider transactions for the past year from Finviz.
• Class Arguments:
• ticker: valid company ticker
• source: choice has no effect, data is from Finviz
• freq: choice has no effect, most recent data is returned

fd = Fundamentals('AAPL')
fd.insider_transactions()

• Returns recent history of analyst ratings from Finviz.
• Class Arguments:
• ticker: valid company ticker
• source: choice has no effect, data is from Finviz
• freq: choice has no effect, most recent data is returned

fd = Fundamentals('AAPL')
fd.analyst_ratings()

• Returns current ESG scores from Sustainalytics published on Yahoo Finance.
• Class Arguments:
• ticker: valid company ticker
• source: choice has no effect, data is from Yahoo Finance
• freq: choice has no effect, most recent data is returned

fd = Fundamentals('AAPL')
fd.esg_score()

• Returns current corporate governance scores from Institutional Shareholder Services (ISS) published on Yahoo Finance.
• Class Arguments:
• ticker: valid company ticker
• source: choice has no effect, data is from Yahoo Finance
• freq: choice has no effect, most recent data is returned

fd = Fundamentals('AAPL')
fd.corporate_governance_score()

• Returns company sector, industry, current number of employees and a company description.
• Class Arguments:
• ticker: valid company ticker
• source: choice has no effect, data is from Yahoo Finance
• freq: choice has no effect

fd = Fundamentals('AAPL')
fd.profile()

• Returns current company executives with name, title, salary, age and their gender.
• Class Arguments:
• ticker: valid company ticker
• source: choice has no effect, data is from Yahoo Finance
• freq: choice has no effect

fd = Fundamentals('AAPL')
fd.executives_info()

The functions below help to retrieve daily historical price data from Yahoo Finance as well as most recent option prices from Yahoo Finance or CBOE.

Furthermore, the historical_futures_contracts function enables a bulk download of historical monthly futures contracts up to the year 2000 for currencies, indices, interest rates and commodities including energy, metals and agricultural contracts. The data is downloaded from www.mrci.com but the data is not completely cleaned (yet).

import finpie.price_data

# Historical price data from Yahoo Finance, most recent option prices from Yahoo Finance and CBOE, and futures prices bulk-download...
# from finpie.price_data import price_data
import finpie

• Returns dataframe with daily historical prices from Yahoo Finance.

historical_prices('AAPL')

• Returns two dataframes for current put and call options from Yahoo Finance.

calls, puts = yahoo_option_chain('AAPL')

• Returns two dataframes for current put and call options from CBOE.

calls, puts = cboe_option_chain('AAPL')

• Returns daily price data for a number of monthly future contracts including open interest of each contract for the given date range.

historical_futures_contracts( pd.date_range('2020-01-01', '2020-09-01') )

• Returns daily price data for a number of monthly future contracts including open interest of each contract for the given date.

futures_prices('2020-01-06')

The functions below retrieve economic data from the OECD nad EIA database.

The available OECD timeseries so far include the OECD composite leading indicators, OECD business surveys, OECD main economic indicators and OECD balance of payments.

The data can be accessed by country or for list of countries and for timeseries specific keyword arguments. Not all timeseries are available for all countries at all frequencies.

For available country codes see here.

from finpie.economic_data import oecd_data # or import finpie

# Example for instantiating class for Australia and the USA at monthly frequency with national currencies
oecd = oecd_data.OecdData( country_code = 'USA', freq = 'M', currency_code = 'NXCU')
# or oecd = finpie.OecdData(...)

# Example for instantiating class for all available countries at quarterly frequency with dollar converted currencies
oecd = oecd_data.OecdData( country_code = 'all', freq = 'Q', currency_code = 'CXCU')
# or oecd = finpie.OecdData(...)

• Returns the OECD composite leading indicator with a given measure. Only monthly data available.
• Subject options:
• (default) amplitude adjusted
• LOLITONO - normalised
• LOLITOTR_STSA - trend restored
• LOLITOTR_GYSA - 12-month rate of change of the trend restored
• BSCICP03 - OECD standardised BCI, amplitude adjusted
• CSCICP03 - OECD standardised CCI, amplitude adjusted
• LORSGPRT - ratio to trend (gdp)
• LORSGPNO - normalised ( gdp )
• LORSGPTD - trend ( gdp )
• LORSGPOR_IXOBSA - original seasonally adjusted (gdp)

oecd = oecd_data.OecdData( country_code = 'USA', freq = 'M' )
oecd.cli(subject = 'amplitude')

• Returns the OECD consumer confidence indicator. Only monthly data available.

oecd = oecd_data.OecdData( country_code = 'USA', freq = 'M' )
oecd.cci()

• Returns the OECD business confidence indicator. Only monthly data available.

oecd = oecd_data.OecdData( country_code = 'USA', freq = 'M' )
oecd.bci()

• Returns the M1 monetary aggregate. Not available for all countries.
• Arguments:
• index = True returns an index, index = False returns level values
• seasonally_adjusted = True returns seasonally adjusted index

oecd = oecd_data.OecdData( country_code = 'all', freq = 'M' )
oecd.monetary_aggregates_m1(index = True, seasonally_adjusted = True)

• Returns the M3 monetary aggregate. Not available for all countries.
• Arguments:
• index = True returns an index, index = False returns level values
• seasonally_adjusted = True returns seasonally adjusted index

oecd = oecd_data.OecdData( country_code = 'all', freq = 'M' )
oecd.monetary_aggregates_m3( index = True, seasonally_adjuted = True )

• Returns interbank interest rates. Not available for all countries.

oecd = oecd_data.OecdData( country_code = 'all', freq = 'M' )
oecd.interbank_rates()

• Returns short-term interest rates. Not avaialable for all countries.

oecd = oecd_data.OecdData( country_code = 'all', freq = 'M' )
oecd.short_term_rates()

• Returns long-term interest rates. Not available for all countries.

oecd = oecd_data.OecdData( country_code = 'all', freq = 'M' )
oecd.long_term_rates()

• Returns aggregate share prices of a given country. Not available for all countries.

oecd = oecd_data.OecdData( country_code = 'all', freq = 'M' )
oecd.all_share_prices()

• Returns aggregate share prices of industrial companies from a given country. Not available for all countries.

oecd = oecd_data.OecdData( country_code = 'all', freq = 'M' )
oecd.share_prices_industrials()

• Returns USD spot exchange rates at end of month/quarter.

oecd = oecd_data.OecdData( country_code = 'all', freq = 'M' )
usd_exchange_rates_spot()

• Returns monthly/quarterly average USD exchange rates.

oecd = oecd_data.OecdData( country_code = 'all', freq = 'M' )
oecd.usd_exchange_rates_average()

• Returns overall real exchange rates.

oecd = oecd_data.OecdData( country_code = 'all', freq = 'M' )
oecd.rer_overall()

• Returns value of exports in national currency or dollar converted, etc..
• Arguments:
• growth = True returns seasonally adjusted growth
• growth = False returns monthly level values in specified currency conversion (national or dollar converted)
• seasonally_adjusted = True returns seasonally adjusted monthly level values in specified currency conversion (national or dollar converted)

oecd = oecd_data.OecdData( country_code = 'all', currency_code = 'CXCU', freq = 'M' )
oecd.exports_value(growth = False, seasonally_adjusted = True)

• Returns value of imports in national currency or dollar converted, etc..
• Arguments:
• growth = True returns seasonally adjusted growth
• growth = False returns monthly level values in specified currency conversion (national or dollar converted)
• seasonally_adjusted = True returns seasonally adjusted monthly level values in specified currency conversion (national or dollar converted)

oecd = oecd_data.OecdData( country_code = 'all', currency_code = 'CXCU', freq = 'M' )
oecd.imports_value(growth = False, seasonally_adjusted = True)

• Returns unemployment rates.

oecd = oecd_data.OecdData( country_code = 'all', freq = 'M' )
oecd.unemployment_rate()

• Returns the consumer price index.
• Arguments:
• growth = True returns yoy growth
• growth = False returns index
• growth = False and seasonally_adjusted = True returns seasonally adjusted index

oecd = oecd_data.OecdData( country_code = 'all', freq = 'M' )
oecd.cpi_total(growth = False, seasonally_adjusted = True)

• Returns the consumer price index for cities.

oecd = oecd_data.OecdData( country_code = 'all', freq = 'M' )
oecd.cpi_city_total()

• Returns non-food and non-energy consumer price index .
• Arguments:
• growth = True returns yoy growth
• growth = False returns index
• growth = False and seasonally_adjusted = True returns seasonally adjusted index

oecd = oecd_data.OecdData( country_code = 'all', freq = 'M' )
oecd.cpi_non_food_non_energy(growth = False, seasonally_adjusted = True)

• Returns consumer price index for energy (fuel, electricity, etc.).
• Arguments:
• growth = True returns yoy growth
• growth = False returns index
• growth = False and seasonally_adjusted = True returns seasonally adjusted index

oecd = oecd_data.OecdData( country_code = 'all', freq = 'M' )
oecd.cpi_energy(growth = False, seasonally_adjusted = True)

• Returns national business tendency survey for given sector.
• Sector arguments:
• (default) retail
• construction
• services
• manufacturing

oecd = oecd_data.OecdData( country_code = 'all', freq = 'M' )
oecd.business_tendency_survey('retail')

• Returns national consumer opinion survey.

oecd = oecd_data.OecdData( country_code = 'all', freq = 'M' )
oecd.consumer_opinion_survey()