add rate limit (by default) about coinbasepro-python HOT 9 CLOSED

gdax-node is having the same discussion coinbase/coinbase-pro-node#61

from coinbasepro-python.

https://docs.gdax.com/#get-historic-rates

The maximum number of data points for a single request is 200 candles. If your selection of start/end time and granularity will result in more than 200 data points, your request will be rejected. If you wish to retrieve fine granularity data over a larger time range, you will need to make multiple requests with new start/end ranges.

from coinbasepro-python.

Right, there's currently no rate-limiting in this project. Have you encountered any 429 responses in your usage? I've found in testing that requests made serially (such as retrieving a full order history) execute just fine since my typical round-trip request time is about 0.5 seconds. I've even tried flooding the public API with multi-threaded requests and didn't get any 429 responses.

If it is an issue for you, it's definitely possible to look into it.

from coinbasepro-python.

I would love to look into this more. Anyone experiencing this, please drop a quick "ME!" or thumps up in this thread :)

from coinbasepro-python.

If it is an issue for you, it's definitely possible to look into it.

I haven't use GDAX-Python yet, I was just researching which library to use.. :)

from coinbasepro-python.

ME! I get the quite frequently.

What I was trying to do is to just get the best bid / ask at any given time. One idea would be to use the feed and generate the best bid / ask yourself (as it is done in order_book.py)
or just send get_product_order_book() in a loop.
The latter one obviously results in rate limits if you don't slow it down.
The first one however has a long round trip (as @acontry pointed out).
Does anyone have an idea how to solve this problem?

from coinbasepro-python.

@jonnylangefeld using one of the websocket feeds is ideal for that use case, no continuous http requests required. If you really want to make a bunch of http requests, GDAX describes their rate limiting as:

"We throttle public endpoints by IP: 3 requests per second, up to 6 requests per second in bursts."

You can do something simple on your end like spacing out your looped requests on a 0.33 second delay. Or if you want to be fancy, route all requests through a rate-limiting algorithm that allows bursting like token bucket algorithm. I found at least one python implementation.

from coinbasepro-python.

I feel like I need to necro this issue... throttling is not possible on paginated endpoints as _send_paginated_message contains a generator that does not check the result for 429 response or rate limit error message. In order to use this library with paged results that method at least should be respectful of rate limits.

` def _send_paginated_message(self, endpoint, params=None):
""" Send API message that results in a paginated response.

    The paginated responses are abstracted away by making API requests on
    demand as the response is iterated over.

    Paginated API messages support 3 additional parameters: `before`,
    `after`, and `limit`. `before` and `after` are mutually exclusive. To
    use them, supply an index value for that endpoint (the field used for
    indexing varies by endpoint - get_fills() uses 'trade_id', for example).
        `before`: Only get data that occurs more recently than index
        `after`: Only get data that occurs further in the past than index
        `limit`: Set amount of data per HTTP response. Default (and
            maximum) of 100.

    Args:
        endpoint (str): Endpoint (to be added to base URL)
        params (Optional[dict]): HTTP request parameters

    Yields:
        dict: API response objects

    """
    if params is None:
        params = dict()
    url = self.url + endpoint
    while True:
        r = self.session.get(url, params=params, auth=self.auth, timeout=30)
        results = r.json()
        for result in results:
            yield result
        # If there are no more pages, we're done. Otherwise update `after`
        # param to get next page.
        # If this request included `before` don't get any more pages - the
        # cbpro API doesn't support multiple pages in that case.
        if not r.headers.get('cb-after') or \
                params.get('before') is not None:
            break
        else:
            params['after'] = r.headers['cb-after']`

from coinbasepro-python.

I agree, I'm hitting this pretty often now on the get_fills endpoint. @danpaquin can we reopen?

from coinbasepro-python.