Code Monkey home page Code Monkey logo

lucit-systems-and-development / unicorn-binance-local-depth-cache Goto Github PK

View Code? Open in Web Editor NEW
36.0 2.0 10.0 15.75 MB

A Python SDK by LUCIT for accessing and managing multiple local Binance order books with Python in a simple, fast, flexible, robust and fully featured way.

Home Page: https://unicorn-binance-local-depth-cache.docs.lucit.tech/

License: Other

Python 95.45% Shell 3.77% Makefile 0.34% Batchfile 0.44%
python binance depthcache orderbook bot-framework unicorn-binance-suite

unicorn-binance-local-depth-cache's Introduction

Get a UNICORN Binance Suite License

Anaconda Release GitHub Release PyPi Release Supported Python Version License PyPi Downloads PyPi Downloads PyPi Downloads PyPI - Status codecov CodeQL Unit Tests Build and Publish GH+PyPi Build and Publish Anaconda Read the Docs Read How To`s Github Telegram Gitter Get Free Professional Support

LUCIT-UBLDC-Banner

UNICORN Binance Local Depth Cache

Description | Live Demo | Installation | How To | Documentation | Examples | Change Log | Wiki | Social | Notifications | Bugs | Contributing |Disclaimer | Commercial Support

A Python SDK by LUCIT for accessing and managing multiple local Binance order books with Python in a simple, fast, flexible, robust and fully functional way.

The organization of the DepthCache takes place in the same asyncio loop as the reception of the websocket data. The full stack of the UBS modules (REST, WebSocket and DepthCache) can be downloaded and installed by PyPi and Anaconda as a Python C extension for maximum performance.

Part of 'UNICORN Binance Suite'.

Get help with the integration of the UNICORN Binance Suite modules!

Get a UNICORN Binance Suite License

To run modules of the UNICORN Binance Suite you need a valid license!

Using a DepthCache

Create a local depth_cache for Binance with just 3 lines of code

from unicorn_binance_local_depth_cache import BinanceLocalDepthCacheManager, DepthCacheOutOfSync

ubldc = BinanceLocalDepthCacheManager(exchange="binance.com", depth_cache_update_interval=100)
ubldc.create_depth_cache("BTCUSDT")

Get the asks and bids

To obtain the complete order book

asks = ubldc.get_asks("BTCUSDT")
bids = ubldc.get_bids("BTCUSDT")

Get the first X elements

asks = ubldc.get_asks("BTCUSDT", limit_count=10)
bids = ubldc.get_bids("BTCUSDT", limit_count=10)

Retain the elements until volume X has been exceeded

asks = ubldc.get_asks("BTCUSDT", threshold_volume=300000)
bids = ubldc.get_bids("BTCUSDT", threshold_volume=300000)

Catch an exception, if the depth_cache is out of sync while accessing its data

try:
    asks = ubldc.get_asks(market="BTCUSDT", limit_count=5, threshold_volume=300000)
    bids = ubldc.get_bids(market="BTCUSDT", limit_count=5, threshold_volume=300000)
except DepthCacheOutOfSync:
    asks = "Out of sync!"
    bids = "Out of sync!"
ubldc.stop_depth_cache("BTCUSDT")

Stop ubldc after usage to avoid memory leaks

When you instantiate UBLDC with with, ubldc.stop_manager() is automatically executed upon exiting the with-block.

with BinanceWebSocketApiManager() as ubldc:
    ubldc.create_depth_cache("BTCUSDT")

Without with, you must explicitly execute ubldc.stop_manager() yourself.

ubldc.stop_manager()

Discover more possibilities.

Description

The Python package UNICORN Binance Local Depth Cache provides local order books for the Binance Exchanges Binance (+Testnet), Binance Futures (+Testnet) and Binance US.

The algorithm of the depth_cache management was designed according to these instructions:

With Create_depth_cache()` the depth_cache is started and initialized, i.e. for each depth_cache that is to be created, a separate asyncio coroutine is inserted into the event loop of the stream. As soon as at least one depth update is received via websocket is received, a REST snapshot is downloaded and the depth updates are applied to it so that it is synchronized in real time. As soon as Once this is done, the status of the cache is set to "synchronous".

Data in the depth_cache can be accessed with 'get_asks()' and 'get_bids()'. If the state of the depth_cache is not synchronous during access, the exception 'DepthCacheOutOfSync' is thrown.

The depth_cache will immediately start an automatic re-initialization if a gap in the UpdateID`s is detected (missing update event) or if the websocket connection is interrupted. As soon as this happens the state of the depth_cache is set to "out of sync" and when accessing the cache the exception 'DepthCacheOutOfSync' is thrown.

Why a local depth_cache?

A local depth_cache is the fastest way to access the current order book depth at any time while transferring as little data as necessary. A REST snapshot takes a lot of time and the amount of data that is transferred is relatively large. Continuous full transmission of the order book via websocket is faster, but the amount of data is huge. A local depth_cache is initialized once with a REST snapshot and then handles Diff. Depth updates applied by the websocket connection. By transferring a small amount of data (only the changes), a local depth_cache is kept in sync in real time and also allows extremely fast (local) access to the data without exceeding the Binance request weight limits.

What are the benefits of the UNICORN Binance Local Depth Cache?

  • Always know if the cache is in sync! If the depth_cache is out of sync, the exception 'DepthCacheOutOfSync' is thrown or ask with is_depth_cache_synchronized().

  • If a depth cache is out of sync it gets refreshed automatically within a few seconds.

  • 100% Websocket auto-reconnect!

  • Supported Exchanges

Exchange Exchange string
Binance binance.com
Binance Testnet binance.com-testnet
Binance USD-M Futures binance.com-futures
Binance USD-M Futures Testnet binance.com-futures-testnet
Binance US binance.us

If you like the project, please star it on GitHub!

Live Demo

This live demo script runs DepthCaches from binance.com-futures and runs on a CCX13 virtual machine of HETZNER CLOUD.

Open live monitor!

live-demo

(Refresh update once a minute!)

Installation and Upgrade

The module requires Python 3.7 and runs smoothly up to and including Python 3.12.

For the PyPy interpreter we offer packages only from Python version 3.9 and higher.

Anaconda packages are available from Python version 3.8 and higher, but only in the latest version!

The current dependencies are listed here.

If you run into errors during the installation take a look here.

Packages are created automatically with GitHub Actions

When a new release is to be created, we start two GitHubActions:

Both start virtual Windows/Linux/Mac servers provided by GitHub in the cloud with preconfigured environments and create the respective compilations and stub files, pack them into wheels and conda packages and then publish them on GitHub, PYPI and Anaconda. This is a transparent method that makes it possible to trace the source code behind a compilation.

A Cython binary, PyPy or source code based CPython wheel of the latest version with pip from PyPI

Our Cython and PyPy Wheels are available on PyPI, these wheels offer significant advantages for Python developers:

  • Performance Boost with Cython Wheels: Cython is a programming language that supplements Python with static typing and C-level performance. By compiling Python code into C, Cython Wheels can significantly enhance the execution speed of Python code, especially in computationally intensive tasks. This means faster runtimes and more efficient processing for users of our package.

  • PyPy Wheels for Enhanced Efficiency: PyPy is an alternative Python interpreter known for its speed and efficiency. It uses Just-In-Time (JIT) compilation, which can dramatically improve the performance of Python code. Our PyPy Wheels are tailored for compatibility with PyPy, allowing users to leverage this speed advantage seamlessly.

Both Cython and PyPy Wheels on PyPI make the installation process simpler and more straightforward. They ensure that you get the optimized version of our package with minimal setup, allowing you to focus on development rather than configuration.

Installation

pip install unicorn-binance-local-depth-cache

Update

pip install unicorn-binance-local-depth-cache --upgrade

A Conda Package of the latest version with conda from Anaconda

The unicorn-binance-local-depth-cache package is also available as a Cython version for the linux-64, osx-64 and win-64 architectures with Conda through the lucit channel.

For optimal compatibility and performance, it is recommended to source the necessary dependencies from the conda-forge channel.

Installation

conda config --add channels conda-forge
conda config --add channels lucit
conda install -c lucit unicorn-binance-local-depth-cache

Update

conda update -c lucit unicorn-binance-local-depth-cache

From source of the latest release with PIP from GitHub

Linux, macOS, ...

Run in bash:

pip install https://github.com/LUCIT-Systems-and-Development/unicorn-binance-local-depth-cache/archive/$(curl -s https://api.github.com/repos/LUCIT-Systems-and-Development/unicorn-binance-local-depth-cache/releases/latest | grep -oP '"tag_name": "\K(.*)(?=")').tar.gz --upgrade

Windows

Use the below command with the version (such as 2.1.1) you determined here:

pip install https://github.com/LUCIT-Systems-and-Development/unicorn-binance-local-depth-cache/archive/2.1.1.tar.gz --upgrade

From the latest source (dev-stage) with PIP from GitHub

This is not a release version and can not be considered to be stable!

pip install https://github.com/LUCIT-Systems-and-Development/unicorn-binance-local-depth-cache/tarball/master --upgrade

Change Log

https://lucit-systems-and-development.github.io/unicorn-binance-local-depth-cache/changelog.html

Documentation

Examples

Howto

Project Homepage

https://www.lucit.tech/unicorn-binance-local-depth-cache.html

Wiki

https://github.com/LUCIT-Systems-and-Development/unicorn-binance-local-depth-cache/wiki

Social

Receive Notifications

To receive notifications on available updates you can watch the repository on GitHub, write your own script with using is_update_available().

Follow us on LinkedIn, X or Facebook!

To receive news (like inspection windows/maintenance) about the Binance API`s subscribe to their telegram groups:

How to report Bugs or suggest Improvements?

List of planned features - click thumbs-up if you need one of them or suggest a new feature!

Before you report a bug, try the latest release. If the issue still exists, provide the error trace, OS and Python version and explain how to reproduce the error. A demo script is appreciated.

If you dont find an issue related to your topic, please open a new issue!

Report a security bug!

Contributing

UNICORN Binance Local Depth Cache is an open source project which welcomes contributions which can be anything from simple documentation fixes and reporting dead links to new features. To contribute follow this guide.

Contributors

Contributors

We love open source!

Disclaimer

This project is for informational purposes only. You should not construe this information or any other material as legal, tax, investment, financial or other advice. Nothing contained herein constitutes a solicitation, recommendation, endorsement or offer by us or any third party provider to buy or sell any securities or other financial instruments in this or any other jurisdiction in which such solicitation or offer would be unlawful under the securities laws of such jurisdiction.

If you intend to use real money, use it at your own risk!

Under no circumstances will we be responsible or liable for any claims, damages, losses, expenses, costs or liabilities of any kind, including but not limited to direct or indirect damages for loss of profits.

Commercial Support

Get professional and fast support

Do you need a developer, operator or consultant? Contact us for a non-binding initial consultation!

unicorn-binance-local-depth-cache's People

Contributors

oliver-zehentleitner avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

unicorn-binance-local-depth-cache's Issues

File conflict during installation

Check this or we will delete your issue. (fill in the checkbox with an X like so: [x])

  • I have searched for other issues with the same problem or similar feature requests.

Select one:

  • Bug

Operating System? (include version)

  • Linux (include flavour) Arch Linux

Exact Python Version?

3.10.2

Pip Version?

21

Description of your issue

When I install the package it seems it has some duplicate files with unicorn-binance-websocket-api, because of the duplicates files it fails to install. Because unicorn-binance-websocket-apiis a requirement for unicorn-binance-local-depth-cacheyou don't need to include duplicates of files already needed.

python-unicorn-binance-local-depth-cache: /usr/lib/python3.10/site-packages/tools/__init__.py exists in filesystem (owned by python-unicorn-binance-websocket-api)
python-unicorn-binance-local-depth-cache: /usr/lib/python3.10/site-packages/tools/__pycache__/__init__.cpython-310.opt-1.pyc exists in filesystem (owned by python-unicorn-binance-websocket-api)
python-unicorn-binance-local-depth-cache: /usr/lib/python3.10/site-packages/tools/__pycache__/__init__.cpython-310.pyc exists in filesystem (owned by python-unicorn-binance-websocket-api)
python-unicorn-binance-local-depth-cache: /usr/lib/python3.10/site-packages/tools/__pycache__/get_used_module_version.cpython-310.opt-1.pyc exists in filesystem (owned by python-unicorn-binance-websocket-api)
python-unicorn-binance-local-depth-cache: /usr/lib/python3.10/site-packages/tools/__pycache__/get_used_module_version.cpython-310.pyc exists in filesystem (owned by python-unicorn-binance-websocket-api)
python-unicorn-binance-local-depth-cache: /usr/lib/python3.10/site-packages/tools/get_used_module_version.py exists in filesystem (owned by python-unicorn-binance-websocket-api)
Errors occurred, no packages were upgraded.

ubra is causing errors during massiv reconnects

Exception in thread Thread-7:
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/home/oliver/.local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/home/oliver/.local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.8/ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.8/ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.8/site-packages/requests/adapters.py", line 440, in send
    resp = conn.urlopen(
  File "/home/oliver/.local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 785, in urlopen
    retries = retries.increment(
  File "/home/oliver/.local/lib/python3.8/site-packages/urllib3/util/retry.py", line 550, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/oliver/.local/lib/python3.8/site-packages/urllib3/packages/six.py", line 769, in reraise
    raise value.with_traceback(tb)
  File "/home/oliver/.local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/home/oliver/.local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/home/oliver/.local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.8/ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.8/ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/PycharmProjects/unicorn-binance-local-depth-cache/unicorn_binance_local_depth_cache/manager.py", line 258, in _process_stream_data
    self._init_depth_cache(market=market.lower())
  File "/home/oliver/PycharmProjects/unicorn-binance-local-depth-cache/unicorn_binance_local_depth_cache/manager.py", line 206, in _init_depth_cache
    order_book = self.ubra.futures_order_book(symbol=market.upper(), limit=1000)
  File "/home/oliver/.local/lib/python3.8/site-packages/unicorn_binance_rest_api/manager.py", line 5502, in futures_order_book
    return self._request_futures_api('get', 'depth', data=params)
  File "/home/oliver/.local/lib/python3.8/site-packages/unicorn_binance_rest_api/manager.py", line 471, in _request_futures_api
    return self._request(method, uri, signed, True, **kwargs)
  File "/home/oliver/.local/lib/python3.8/site-packages/unicorn_binance_rest_api/manager.py", line 450, in _request
    self.response = getattr(self.session, method)(uri, **kwargs)
  File "/home/oliver/.local/lib/python3.8/site-packages/requests/sessions.py", line 542, in get
    return self.request('GET', url, **kwargs)
  File "/home/oliver/.local/lib/python3.8/site-packages/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/oliver/.local/lib/python3.8/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/home/oliver/.local/lib/python3.8/site-packages/requests/adapters.py", line 501, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

dictionary changed size during iteration in manager._sort_depth_cache

Check this or we will delete your issue. (fill in the checkbox with an X like so: [x])

  • [x ] I have searched for other issues with the same problem or similar feature requests.

Select one:

  • [x ] Bug
  • Feature Request
  • Technical Help
  • Other

Environment

  • Are you using the module on a VPS or other Cloud hosting?
  • Are you using the module on a Raspberry Pi?

What kind of internet connection do you have?

cable

Average system load (CPU)

70%

Hardware specification

Intel© Core™ i3-2120T CPU @ 2.60GHz × 2, 
10Gib 

Operating System? (include version)

  • macOS
  • Windows
  • [ x] Linux (Mint)

Which endpoint do you connect?

 binance.com

Python Version Requirement

  • [ x] I am using Python 3.7 or above

Exact Python Version?

3.8.10

Pip Version?

3.8.10

Dependencies

Run pip list > pip_list.txt and upload the file.
pip_list.txt

UNICORN Binance Local Depth Cache Version?

Did you upgrade to the latest release version with `pip install unicorn-binance-local-depth-cache --upgrade`? No

unicorn-binance-local-depth-cache-0.1.0

Description of your issue

[example_depthcache.py.log](https://github.com/LUCIT-Systems-and-Development/unicorn-binance-local-depth-cache/files/8186753/example_depthcache.py.log)

Running the next file:

example_depthcache_2.py.txt

Traceback (most recent call last):
File "/test/unicorn/example_depthcache.py", line 61, in
print(market + f" Top 10 asks: {ubldc.get_asks(market=market)[:10]}")
File "venv2/lib/python3.8/site-packages/unicorn_binance_local_depth_cache/manager.py", line 399, in get_asks
return self._sort_depth_cache(self.depth_caches[market.lower()]['asks'], reverse=False)
File "/venv2/lib/python3.8/site-packages/unicorn_binance_local_depth_cache/manager.py", line 321, in _sort_depth_cache
new_items = [[float(price), float(quantity)] for price, quantity in items.items()]
File /venv2/lib/python3.8/site-packages/unicorn_binance_local_depth_cache/manager.py", line 321, in
new_items = [[float(price), float(quantity)] for price, quantity in items.items()]
RuntimeError: dictionary changed size during iteration

NO GIL

Is your feature request related to a problem? Please describe.

In Cython it is possible to free subroutines from the GIL restrictions.

I will test this with the CPU intensive methods _sort_depth_cache() and _apply_updates().

Describe the solution you'd like.

No response

Describe alternatives you've considered

No response

Additional context

No response

Slow data retrieval

Is your feature request related to a problem? Please describe.
Depth data retrieval is 20-50ms late compared to the binance api ThreadedDepthCacheManager (from sammchardy).
This is far from negligible for a 100ms update rate.
Tested with binance spot 'BTCUSDT' get_bids and get_asks.

Describe the solution you'd like
Be at par with other available solutions.

Describe alternatives you've considered
I have considered the library https://github.com/sammchardy/python-binance

get_last_update_time()

Is your feature request related to a problem? Please describe.

Track the last update time and make it accessable.

Describe the solution you'd like.

No response

Describe alternatives you've considered

No response

Additional context

No response

Query order book data in the form of candlesticks?

Is your feature request related to a problem? Please describe.

NA

Describe the solution you'd like.

Functionality to aggregate tick data into the form of candlesticks such that it matches the format and frequency of Kline data from other binance web socket data.

Describe alternatives you've considered

No response

Additional context

No response

BinanceLocalDepthCacheManager._init_depth_cache() - Can not download order_book snapshot for the depth_cache

Version of this library.

0.7.2

Solution to Issue cannot be found in the documentation or other Issues and also occurs in the latest version of this library.

  • I checked the documentation and other Issues. I am using the latest version of this library.

Hardware?

Local server/workstation

Operating System?

Linux

Python version?

Python3.9

Installed packages

Package                                   Version
----------------------------------------- -------------------
absl-py                                   0.15.0
aniso8601                                 9.0.1
argon2-cffi                               21.3.0
argon2-cffi-bindings                      21.2.0
asttokens                                 2.0.5
astunparse                                1.6.3
asyncio                                   3.4.3
attrs                                     21.2.0
auto-sklearn                              0.14.5
autokeras                                 1.0.16.post1
backcall                                  0.2.0
black                                     22.1.0
bleach                                    4.1.0
cachetools                                5.0.0
certifi                                   2021.10.8
cffi                                      1.15.0
charset-normalizer                        2.0.8
cheroot                                   8.5.2
click                                     8.0.3
cloudpickle                               2.0.0
colorama                                  0.4.4
configparser                              5.1.0
ConfigSpace                               0.4.20
cryptography                              36.0.0
cycler                                    0.11.0
Cython                                    0.29.27
dask                                      2022.1.0
dateparser                                1.1.0
debugpy                                   1.5.1
decorator                                 5.1.1
defusedxml                                0.7.1
distributed                               2022.1.0
distro                                    1.6.0
emcee                                     3.1.1
entrypoints                               0.3
eventhandler                              1.2.1
executing                                 0.8.2
FLAML                                     0.9.5
Flask                                     2.0.2
Flask-RESTful                             0.3.9
flatbuffers                               1.12
fonttools                                 4.28.5
fsspec                                    2022.1.0
gast                                      0.4.0
google-auth                               2.6.0
google-auth-oauthlib                      0.4.6
google-pasta                              0.2.0
grpcio                                    1.34.1
h5py                                      3.1.0
HeapDict                                  1.0.1
idna                                      3.3
importlib-metadata                        4.10.1
investpy                                  1.0.7
ipykernel                                 6.7.0
ipython                                   7.31.1
ipython-genutils                          0.2.0
itsdangerous                              2.0.1
jaraco.functools                          3.4.0
jedi                                      0.18.1
Jinja2                                    3.0.3
joblib                                    1.1.0
jsonschema                                4.4.0
jupyter-client                            7.1.2
jupyter-core                              4.9.1
jupyterlab-pygments                       0.1.2
keras-nightly                             2.5.0.dev2021032900
Keras-Preprocessing                       1.1.2
keras-tuner                               1.0.4
kiwisolver                                1.3.2
kt-legacy                                 1.0.4
liac-arff                                 2.5.0
lightgbm                                  3.3.2
llvmlite                                  0.38.0
locket                                    0.2.1
lxml                                      4.6.4
Markdown                                  3.3.6
MarkupSafe                                2.0.1
matplotlib                                3.5.1
matplotlib-inline                         0.1.3
matrixprofile                             1.1.10
mistune                                   0.8.4
more-itertools                            8.12.0
msgpack                                   1.0.3
mypy-extensions                           0.4.3
nbclient                                  0.5.10
nbconvert                                 6.4.1
nbformat                                  5.1.3
nest-asyncio                              1.5.4
networkx                                  2.6.3
notebook                                  6.4.8
numba                                     0.55.0
numpy                                     1.21.5
oauthlib                                  3.2.0
opt-einsum                                3.3.0
packaging                                 21.3
pandas                                    1.3.4
pandocfilters                             1.5.0
parso                                     0.8.3
partd                                     1.2.0
pathlib                                   1.0.1
pathspec                                  0.9.0
patsy                                     0.5.2
pexpect                                   4.8.0
pickleshare                               0.7.5
pika                                      1.2.0
pika-stubs                                0.1.3
Pillow                                    9.0.0
pip                                       22.1
pipelineprofiler                          0.1.17
platformdirs                              2.4.1
prometheus-client                         0.13.1
prompt-toolkit                            3.0.26
protobuf                                  3.19.4
psutil                                    5.8.0
ptyprocess                                0.7.0
pure-eval                                 0.2.2
pyaes                                     1.6.1
pyasn1                                    0.4.8
pyasn1-modules                            0.2.8
pycparser                                 2.21
Pygments                                  2.11.2
pynisher                                  0.6.4
pyOpenSSL                                 21.0.0
pyparsing                                 3.0.7
pyproj                                    3.3.0
pyrfr                                     0.8.2
pyrsistent                                0.18.1
pyshp                                     2.2.0
pysqlite3                                 0.4.6
python-dateutil                           2.8.2
pytz                                      2021.3
pytz-deprecation-shim                     0.1.0.post0
PyYAML                                    6.0
pyzmq                                     22.3.0
regex                                     2021.11.10
requests                                  2.26.0
requests-oauthlib                         1.3.1
rsa                                       4.8
scikit-learn                              0.24.2
scipy                                     1.7.3
Send2Trash                                1.8.0
service-identity                          21.1.0
setuptools                                59.4.0
simdkalman                                1.0.2
six                                       1.15.0
smac                                      1.1.1
sortedcontainers                          2.4.0
spark                                     0.2.1
stack-data                                0.1.4
statsmodels                               0.13.1
stockstats                                0.4.1
stumpy                                    1.10.2
talipp                                    1.7.0
tblib                                     1.7.0
Telethon                                  1.24.0
tensorboard                               2.8.0
tensorboard-data-server                   0.6.1
tensorboard-plugin-wit                    1.8.1
tensorflow                                2.5.2
tensorflow-estimator                      2.5.0
termcolor                                 1.1.0
terminado                                 0.13.1
testpath                                  0.5.0
threadpoolctl                             3.0.0
tomli                                     2.0.0
toolz                                     0.11.2
tornado                                   6.1
tqdm                                      4.62.3
traitlets                                 5.1.1
tsfresh                                   0.19.0
tsmoothie                                 1.0.4
typing-extensions                         3.7.4.3
tzdata                                    2021.5
tzlocal                                   4.1
ujson                                     4.3.0
unicorn-binance-local-depth-cache         0.7.2
unicorn-binance-rest-api                  1.5.0
unicorn-binance-trailing-stop-loss-engine 0.0.0
unicorn-binance-websocket-api             1.41.0
unicorn-fy                                0.11.0
Unidecode                                 1.3.2
urllib3                                   1.26.7
wcwidth                                   0.2.5
webencodings                              0.5.1
websocket-client                          1.2.1
websockets                                10.3
Werkzeug                                  2.0.2
wheel                                     0.37.0
wrapt                                     1.12.1
xgboost                                   1.3.3
zict                                      2.0.0
zipp                                      3.7.0

Logging output

Sorry, don't know where is the log file

Used endpoint?

binance.com-futures

Issue

Hello.
Testing the library with this code:

-------------------------------------------------------

future_client = BinanceRestApiManager(exchange="binance.com-futures",
api_key=api_key,
api_secret=api_secret)

bw_api_manager = BinanceWebSocketApiManager(exchange="binance.com-futures",
high_performance=True, enable_stream_signal_buffer=True)
ubldc = BinanceLocalDepthCacheManager(exchange="binance.com-futures", ubwa_manager=bw_api_manager)
markets = ['BNBBUSD', 'ETHBUSD', 'GMTBUSD', 'XRPBUSD', 'SOLBUSD', 'ADABUSD', 'DOGEBUSD']
ubldc.create_depth_cache(markets=markets, refresh_interval=5)

while True:
stream_data = bw_api_manager.pop_stream_data_from_stream_buffer()
kline = json.loads(stream_data)
symbol = kline['data']['s']

# doing things with the stream

if condition(symbol): # This condition is True more or less 6 time in 10',  in less than a 10'' 
       asks = ubldc.get_asks(market=market)[:10]
       bids = ubldc.get_bids(market=market)[:10]

--------------

Before an hour the next error is showing:

BinanceLocalDepthCacheManager._init_depth_cache() - Can not download order_book snapshot for the depth_cache with market ethbusd - requests.exceptions.ReadTimeout - error_msg: HTTPSConnectionPool(host='fapi.binance.com', port=443): Read timed out. (read timeout=10)
Exception in thread Thread-5:
Traceback (most recent call last):
File "/usr/lib/python3.9/threading.py", line 973, in _bootstrap_inner
self.run()
File "/usr/lib/python3.9/threading.py", line 910, in run
self._target(*self._args, **self._kwargs)
File "/home/workspace/crypto/futuress/venv/lib/python3.9/site-packages/unicorn_binance_local_depth_cache/manager.py", line 372, in _process_stream_data
if not self._init_depth_cache(market=market):
File "/home/workspace/crypto/futures/venv/lib/python3.9/site-packages/unicorn_binance_local_depth_cache/manager.py", line 326, in _init_depth_cache
self.depth_caches[market]['last_update_id'] = int(order_book['lastUpdateId'])
TypeError: 'NoneType' object is not subscriptable

Ip ban when using ‘create_depth_cache’ with many symbols

Is your feature request related to a problem? Please describe.

When passing list with many symbols in create_depth_cache method we getting ip ban, because ubldc sent rest to binance by each symbol.

Describe the solution you'd like.

No response

Describe alternatives you've considered

No response

Additional context

No response

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.