streetyoga / atapi Goto Github PK
View Code? Open in Web Editor NEWBacktest, forward test and apply trading algorithms and strategies on binance spot, futures and nft market and other exchanges, including arbitrage.
License: MIT License
Backtest, forward test and apply trading algorithms and strategies on binance spot, futures and nft market and other exchanges, including arbitrage.
License: MIT License
# 'q' quits also console, solution needed
def menu():
while user_input := input(USER_CHOICE):
{'b': print_best_books,
'c': print_cheapest_books,
'n': get_next_book,
'q': sys.exit
}.get(user_input, lambda: print('Invalid Value'))()
You can do websockets in Flask too: Flask SocketIO
Flasks own request
instance of Request class and requests
Rewrite of cmd to client that gets data from API endpoints instead of importing Algo class.
API documentation in get request at root, i.e. api.street.yoga/
Returns all endpoints and servertime from class, but split it up to appropriate resource classes and save to database.
check postman Code snippets for http.client or requests payload
See this list of http-clients, it includes asynchronious libraries and even one that offers HTTP 2.0 support.
Using one of the asynchronious ones is a strong consideration.
Websockets under Windows: pyOpenSSL expects trusted root set from twisted but gets none:
In PowerShell:
binance/binance-connector-python#57 (comment)
PS C:\> $env:SSL_CERT_FILE=(py -m certifi); py yourscript.py
In Script:
binance/binance-connector-python#57 (comment)
import os, certifi, win32api
from binance.websocket.spot.websocket_client \
import SpotWebsocketClient as WebsocketClient
os.environ['SSL_CERT_FILE'] = certifi.where()
ws_client = WebsocketClient()
def message_handler(message):
print(message)
ws_client.start()
ws_client.mini_ticker(
symbol='bnbusdt',
id=1,
callback=message_handler,
)
# Combine selected streams
ws_client.instant_subscribe(
stream=['bnbusdt@bookTicker', 'ethusdt@bookTicker'],
callback=message_handler,
)
win32api.SetConsoleCtrlHandler(lambda _: ws_client.stop(), True)
Cryptographic Authority wants to kill pyopenssl, avoid, as well as frameworks that depend on it, like twisted
python-trio/trio#1145 (comment)
Removed binance-connector because being dependant on twisted
Pyramid and Django probably overkill
Evaluation has to be made which plugins will be future proof, all candidates are updated regularly.
Research future prospects on new functionality like trading NFT's...
Binance public API connector
Advantages: Official from Binance, easy to understand
Disadvantages: lightweight More work has to be done. Only Spot trading, no futures, but could be built upon...
binance API docu
This Documentation includes info about NFT endpoints:
API Documentation Site
python-binance
Advantages: Very mature and robust, almost all features from binance API implemented
Disadvantages: Third party
CCXT
Advantages: Supports a streamlined API for hundreds of exchanges, only small adjustments for ticker-names needed,
futureproof for arbitrage trading
Disadvantages: may be slow with implementing new features like NFT trades on binance
Name could be changed to ata
- Alogorithmic Trading API, or atapi
, (like the AT Attachment Packet Interface)
Considerations:
short names like ata
could mean less typing , but something like import atapi as at
could also work...
split into methods so that not everything is run alltogether.
instead of having two methods, return dataframe as tuple before and after creating column. and access by index.
Exact same configuration of test project passes test, by collecting __init__.py
Some more confrontation with pytest is needed
{"key" : "your_api_key","secret":"your_secret_key"}
export BINANCE_API_KEY=<your_api_key>
export BINANCE_API_SECRET=<your_secret_key>
Caches results forever, check all methods and eventually replace with one of this options:
classmethod not needad as class is instantiated anyway for interpreter access with atapi.algo. So no problem that Python 3.11 removed chaining classmethod descriptors has been removed. Initial loading time slower without? Could be rate limiting because of testing cli and module at same time. Time it and check if true.
@functools.lru_cache(maxsize=2) # caches 2 calls
from cachetools import cached, TTLCache
@cached(cache=TTLCache(maxsize=2, ttl=900)) # caches for 15min.
Check if setter property could be useful, i.e.:
@property
def assets_close(self):
return ...
@assets_close.setter
def assets_close(self, ...):
...
assets_close = ...
Other means to to speed up code execution:
Numbas jit decorator:
from numba import jit
@jit(nopython=True)
Check all formulas and implement how to calculate with those DataFrames.
For now I limited calculations with [:3] to the first 3 symbols as those have no NaN values.
Possible solutions:
For annual probabilistic sharpe ratio, a dataframe sample of yearly returns (yearly log returns) over at least 2 years is needed:
Algo().assets_close().resample('Y')
Consider daily/weekly/monthly returns.
pct_change()
returns to log returns:
np.log1p(df['Yearly Returns']
@staticmethod
def psr(df, benchmark=0):
sharpe = stats['Sharpe']
skew = scipy.stats.skew(df['Yearly Returns'])
kurtosis = scipy.stats.kurtosis(df['Yearly Returns'])
n = len(df)
sigma_sr = np.sqrt(
(1/(n-1)) * (1 + 0.5*sharpe**2 - skew*sharpe+(kurtosis/4)*sharpe**2))
ratio = (sharpe - benchmark) / sigma_sr
psr = scipy.stats.norm.cdf(ratio)
return psr
drop and set as multi row index, now i select with .iloc[:, 6]
as they are of no use atm.
Mainnet balance still a long way to go.
API keys not needed as a lot of work has to be done with statistics.
Instructions on how to setup a your API key for the spot testnet:
Export as environment variables:
Bash/Sh/Zsh:
export BINANCE_API_KEY='your_api_key'
export BINANCE_API_SECRET='your_secret_key'
# Check
echo $BINANCE_API_KEY
echo $BINANCE_API_SECRET
Powershell:
$env:BINANCE_API_KEY = 'your_api_key'
$env:BINANCE_API_SECRET = 'your_secret_key'
# Check
$env:BINANCE_API_KEY
$env:BINANCE_API_KEY
from comp/algo.py:
@staticmethod
def balance():
"""Balance and kline fields for selected assets."""
return pd.json_normalize(client_test.account()['balances'])
from __main__.py
@staticmethod
def do_balance(arg):
"""Balance and kline fields for selected assets."""
table = Table('Assets Balance')
table.add_row(algo_class.balance().astype({'free': 'float', 'locked': 'float'}).to_string(
float_format=lambda _: '{:.2f}'.format(_), index=False))
console.print(table)
@Gooey
decorator, works with argparseAll annualized stats don't make sense on testnet, as testnet resets once a month.
Consideration to switch to api.binance.com for data that does not need API key.
Only sums up symbols in use, change to all polled symbols from binance.
def sortino_ratio(df, threshold=0, rfr):
"""Annual Sortino Ratio
downside = df[df < treshold]
std = downside.std()
_sortino_ratio = (self.mean_returns() - rfr) / std
return sortino_ratio * np.sqrt(TD)
Originally posted by miron April 24, 2022
Evaluation has to be made which plugins will be future proof, all candidates are updated regularly.
Research future prospects on new functionality like trading NFT's...
Binance public API connector
Advantages: Official from Binance, easy to understand
Disadvantages: lightweight More work has to be done. Only Spot trading, no futures, but could be built upon...
binance API docu
This Documentation includes info about NFT endpoints:
API Documentation Site
python-binance
Advantages: Very mature and robust, almost all features from binance API implemented
Disadvantages: Third party
CCXT
Advantages: Supports a streamlined API for hundreds of exchanges, only small adjustments for ticker-names needed,
futureproof for arbitrage trading
Disadvantages: may be slow with implementing new features like NFT trades on binance
As bare minimum it should include:
Table looks good.
investigate coins with 0 supply and remove.
remove NaN fiat rows.
symbols and marketcap limited with [:3] due to formulas having to be adapted for NaN values.
This script can do it but needs elevated privileges:
import time
import os
try:
import ntplib
client = ntplib.NTPClient()
response = client.request('pool.ntp.org')
os.system('date ' + time.strftime('%m%d%H%M%Y.%S',time.localtime(response.tx_time)))
except:
print('Could not sync with time server.')
print('Done.')
import pandas as pd
import numpy as np
pd.DataFrame.rolling(window = 7, min_periods = 7).mean()
pd.DataFrame.rolling(window = 25).mean()
pd.DataFrame.rolling(window = 99).mean()
# Common Movin Averages:
price.rolling(window = 50)
price.rolling(window = 200)
price.rolling(window = 5)
price.rolling(window = 10)
price.rolling(window = 20)
price.rolling(window = 100)
# Fibonacci:
price.rolling(window = (5, 8, 13, 21 ...) )
month_ret = pd.DataFrame.resample('M', kind = 'period').last().pct_change().dropna()
for years in [1, 3, 5, 10, 20]:
# Anualized Mean Returns For Last ... Years.
pd.DataFrame['month_ret'].rolling(years * 12).mean() * 12
# Anualized Risk For Last ... Years.
pd.DataFrame['month_ret'].rolling(years * 12).std() * np.sqrt(12)
price.ewm(span = 10, min_periods= 10).mean()
price.expanding().mean()
returns.corr()
returns['BTC'].rolling(36).corr(returns['ETH'])
mean_log_ret = np.log(df / df.shift(1)).dropna().mean()
I.e. price is 130, 110, 120, 90, 80:
returns are -15%, 9%, -25%, -11%
mean: -12.13%
# Actual Price
investment * np.exp(periods * mean_log_ret)
130 * np.exp(4*-0.1213) = 80
[(1+R1)(1+R2 )(1+R3)β¦*(1+Rn)]**(1/n)-1
where:
R=Return
n=Count of the numbers in the series
I.e. returns are respectively
30%, 10%, 20%, -10%, and -80%
(1.3 *1.1 * 1.2 * 0.9 * 0.8)**(1/5)-1
The geometric average annual return == 4.3%
AF class and ATFShell into seperate file, merge if possible.
py -m setuptools_scm
:
Automatically increments to next version, dev2, dev3, for every push, until you tag it for release (git tag v0.1.3)
(0.1.2.dev2+g8d01dfa), needs to be resolved before pushing new build to pypi
At a later stage, Jupyter Notebooks will be abandoned and the script will be converted to a Python module.
There will be a transitioning time, where the convenient Jupyter interface will be kept, but afterwards development will focus on pleasant functionality in Python's REPL, possibly with rich support, kind of like shutil
Object oriented loading of modules, calling various algorityms.
requirements.txt provided, but maybe at the begining dependencies as %pip install pandas
for the notebook
Everyhing should be nicely grouped as atapi.algo.
and within other future instance classes.
Seperated data fetch into own class.
Binance has no market cap data through their public API.
Custom requests query has to be written.
Here is a workaround:
Howto Calculate Market Cap Data with Binance API
Originally posted by @miron in #16 (comment)
Normaly i would do:
from binance.spot import Spot as Client
client = Client()
exchange_info = client.exchange_info()
symbols = set()
for s in exchange_info['symbols']:
if s['symbol'].endswith('USDT'):
symbols.add(s['symbol'])
But as I already use the circulating supply data, I will go with that. it has 80 pairs less, which seem only be some leveraged and very exotic tokens pairs
Different testsuites to consider:
Write test conditions for every single method/function accessed in the cmd-interface, saves time in the long run
Fix.
some ideas:
name = "user"
say = input("Say: ")
sys.stdout.write("\033[F")
print (name, "said:", say)
from getpass import getpass
name = "user"
say = getpass(prompt="")
print (name, "said:",say)
For consistency, all other outputs have to be sorted.
Not sure by which criteria:
you have documented stats, correlation, returns but what about the rest?
clearer indication where to find thouroughfull documentation
indicate that excessive documentation can be found right in the interface.
Documentation of endpoints at / API call
what are the inputs we need to feed to the API for each method
CLI (line interpreter) on top of the JSON API,
calculations: Algo(),
Flask (or other framework) API logic: API(), Data().
To get started, you need to have a clear idea of what to test
when you present your project, specify exactly what you need to be tested.
Portfolio Maker, Crypto Portfolio Maker, CPM
project to categorize cryptocurencies
A tool that uses moderm portfolio theory that helps to categorize cryptocurencies based on those metrics.
Rebranding as crypto lexicon that educates thoroughly but in nontechnical terms about those properties and builds your portfolio in a short time, but long enough to make an informed decision.
Pulls data from Binance and calculates modern portfolio theory metrics like alpha, beta, sharpe ratio, annualized risk/return, categorizes them into defensive, non-cyclical coins according to market needs,..
Feedback from professionals is very important, also from laypersons not knowing anything but would like to invest a little, learn where to start, educate themselves just enough to make an informed decision, all in one interface.
Binance is too complicated for majority.
Coinpanion tries, but:
Focus on requests and binance, ccxt for arbitrage trading with support over hundred exchanges is too much, trading comes later but in simple form just buying the uncorrelated portfolio, one click solution, i.e. invest 100$, diversify in 10 coins a 10$.
There are too many trading frameworks and TA , see forks, QuantConnect i.e. works even with Binance
A simple program for people who don't know anything about crypto coughing up 10 sorted different coins sharing the same universe, by specific criteria of which you choose ONE.
Oracle which asks you about some properties, multiple choice, and then coughs up the top 10-15 with that properties. Also educate about the importance of diversification, so recommend other uncorrelated properties. Build up to a portfolio of 10-15 uncorrelated crypto currencies.
Sort for the best weights, sharpe, risk, returns,
Categorize by consensus mechanisms like POW, POS, DPOS, PTOS, DeFi, NFT, ERC-20, BEP-20, eosio.token...
I wanted to take a list simple list of crypto pairs but could understand how i would input them
so no, did not proceed to test
make people interested enough to install that damn thing :)
I think you should make a βHow toβ step by step in readme
Emphasize the availability of the CLI help with meaningful examples
Emphasize the availability of documentation in the WIKI, or find a less hidden solution.
Make the readme more interesting, less details, more up to go and how to use it quick. Simple and catchy elevator pitch.
TI (coarse filter needs to be implemented before expensive computation)
above 200 day SMA
NVT ratio (Network valute to transaction, MarcetCap/24hVolume, P/E-like metric for Crypto)
10-15 good uncorrelated-return streams (Ray Dalio)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.