Code Monkey home page Code Monkey logo

cryptory's Introduction

cryptory

Retrieve historical cryptocurrency and other related data.

cryptory integrates various sources of historical crypto data, so that you can perform analysis and build models without having to worry about knowing different packages and APIs. Current data sources include:

  • Daily historical prices
  • Additional cryptocurrency information (transaction fees, active adressess, etc.)
  • Reddit metrics (e.g. subscriber growth)
  • Google Trends (via Pytrends)
  • Stock market
  • Foreign exchange rates
  • Commodity prices

Installation

$ pip install cryptory

Compatibility

  • Python 2.7+
  • Python 3

Dependencies

  • pandas>=0.23.0
  • numpy>=1.14.0
  • pytrends>=4.4.0
  • beautifulsoup4>=4.0.0

How to Use

Consult the documentation help(Cryptory) for more information on its usage.

Basic Usage

# load package
from cryptory import Cryptory

# initialise object 
# pull data from start of 2017 to present day
my_cryptory = Cryptory(from_date = "2017-01-01")

# get historical bitcoin prices from coinmarketcap
my_cryptory.extract_coinmarketcap("bitcoin")
# get daily subscriber numbers to the bitcoin reddit page
my_cryptory.extract_reddit_metrics(subreddit="bitcoin",
                                    metric="total-subscribers")
# google trends- bitcoin search results
my_cryptory.get_google_trends(kw_list=["bitcoin"])
# dow jones price (market code from yahoo finance)
my_cryptory.get_stock_prices(market="%5EDJI")
# USD/EUR exchange rate
my_cryptory.get_exchange_rates(from_currency="USD", to_currency="EUR")
# get historical commodity prices
my_cryptory.get_metal_prices()

Advanced Usage

As all cryptory methods return a pandas dataframe, it's relatively easy to combine results and perform more complex calculations.

# generate price correlation matrix
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt

all_coins_df = my_cryptory.extract_bitinfocharts("btc")
# coins of interest
bitinfocoins = ["btc", "eth", "xrp", "bch", "ltc", "dash", "xmr", "doge"]
for coin in bitinfocoins[1:]:
    all_coins_df = all_coins_df.merge(my_cryptory.extract_bitinfocharts(coin), on="date", how="left")
# date column not need for upcoming calculations
all_coins_df = all_coins_df.drop('date', axis=1)
corr = all_coins_df.pct_change().corr(method='pearson')
fig, ax = plt.subplots(figsize=(7,5))  
sns.heatmap(corr, 
            xticklabels=[col.replace("_price", "") for col in corr.columns.values],
            yticklabels=[col.replace("_price", "") for col in corr.columns.values],
            annot_kws={"size": 16})
plt.show()

png

# overlay bitcoin price and google searches for bitcoin
btc_google = my_cryptory.get_google_trends(kw_list=['bitcoin']).merge(
    my_cryptory.extract_coinmarketcap('bitcoin')[['date','close']], 
    on='date', how='inner')

# need to scale columns (min-max scaling)
btc_google[['bitcoin','close']] = (
        btc_google[['bitcoin', 'close']]-btc_google[['bitcoin', 'close']].min())/(
        btc_google[['bitcoin', 'close']].max()-btc_google[['bitcoin', 'close']].min())

fig, ax1 = plt.subplots(1, 1, figsize=(9, 3))
ax1.set_xticks([datetime.date(j,i,1) for i in range(1,13,2) for j in range(2017,2019)])
ax1.set_xticklabels([datetime.date(j,i,1).strftime('%b %d %Y') 
                     for i in range(1,13,2) for j in range(2017,2019)])
ax1.plot(btc_google['date'].astype(datetime.datetime),
             btc_google['close'], label='bitcoin', color='#FF9900')
ax1.plot(btc_google['date'].astype(datetime.datetime),
             btc_google['bitcoin'], label="bitcoin (google search)", color='#4885ed')
ax1.legend(bbox_to_anchor=(0.1, 1), loc=2, borderaxespad=0., ncol=2, prop={'size': 14})
plt.show()

png

Issues & Suggestions

cryptory relies quite strongly on scraping, which means that it can break quite easily. If you spot something not working, then raise an issue. Also, if you have any suggestions or criticism, you can also raise an issue.

cryptory's People

Contributors

dashee87 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cryptory's Issues

Colab and Anaconda installation issue.

I receive the error message "ERROR: Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-wj7tq73d/cryptory/" when trying to pip install cryptory through Colab (Linux) or a virtual anaconda environment (Windows 10). I am using the "pip install cryptory" command. I have tried updating setuptools and pip without any luck.

timeout: The read operation timed out

Suddenly geting the above mentiones error when calling
my_cryptory.get_stock_prices("GOOG")
for example.

Any ideas how to resolve it? It used to work fine and no changes have been made.
Thanks!

Gather hourly data

Is it possible to collect historical hourly data on currencies? It would be great

Name Error

in line 109 'label' should be 'col_label'

Coin Market Cap throwing 404

I believe coin market cap changed how URL params are parsed. Current code base is not working.

line 129 in cryptory.py:
output = pd.read_html("https://coinmarketcap.com/currencies/{}/historical-data/?start={}&end={}".format( coin, self.from_date.replace("-", ""), self.to_date.replace("-", "")))[0]

Error

site-packages/pandas/compat/init.py", line 404, in raise_with_traceback
raise exc.with_traceback(traceback)
urllib.error.HTTPError: HTTP Error 404: NOT FOUND

In case of error do not return dataframe

Hi there
just a small suggestion. Once an error occured I wouldn't return a DataFrame with error but more like throw custom exception or return False/None.

It's much easier to catch that instead of look into the DataFrame.
In other words once I get DataFrame I want to have data.

Thank you.

Google trends throwing 429

I'm sorry to report the Google Trends function isn't working.

It looks like Google have got fed up with PyTrends - it keeps returning an error 429 (too many requests) even with a small request payload. :(

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.