betcode-org / betfair Goto Github PK
View Code? Open in Web Editor NEWbetfairlightweight - Betfair API-NG python wrapper (with streaming)
License: MIT License
betfairlightweight - Betfair API-NG python wrapper (with streaming)
License: MIT License
After reviewing other wrappers looking to change apiclient to be similar to twilio:
Enables the wrapper to more closely resemble the API as well as removing the spaghetti code that is apimethod. This will of course break everything that is built using this wrapper, welcome any comments.
Similar to issue #69.
Contrary to the betfair documentation, the returned json of a placeOrders call won't have a persistenceType attribute if the timeinForce of the submitted bet is FILL_OR_KILL.
When building the resources from a successful place_orders call, building the LimitOrder resource for a PlaceOrderInstructionReport will fail:
https://github.com/liampauling/betfairlightweight/blob/master/betfairlightweight/resources/bettingresources.py#L593
{u'timeInForce': u'FILL_OR_KILL', u'price': 1.4, u'size': 5.0}
File "/home/anon/.virtualenvs/anon-app/local/lib/python2.7/site-packages/betfairlightweight/endpoints/betting.py", line 299, in place_orders
return self.process_response(response, resources.PlaceOrders, elapsed_time, lightweight)
File "/home/anon/.virtualenvs/anon-app/local/lib/python2.7/site-packages/betfairlightweight/endpoints/baseendpoint.py", line 102, in process_response
return resource(elapsed_time=elapsed_time, **result)
File "/home/anon/.virtualenvs/anon-app/local/lib/python2.7/site-packages/betfairlightweight/resources/bettingresources.py", line 664, in __init__
PlaceOrderInstructionReports(**i) for i in kwargs.get('instructionReports')
File "/home/anon/.virtualenvs/anon-app/local/lib/python2.7/site-packages/betfairlightweight/resources/bettingresources.py", line 644, in __init__
self.instruction = PlaceOrderInstruction(**instruction) if instruction else None
File "/home/anon/.virtualenvs/anon-app/local/lib/python2.7/site-packages/betfairlightweight/resources/bettingresources.py", line 621, in __init__
self.order = LimitOrder(**limitOrder)
TypeError: __init__() takes at least 4 arguments (4 given)
Add listRunnerBook to .betting
http://docs.developer.betfair.com/docs/display/1smk3cen4v3lu3yomq5qye0ni/listRunnerBook
At the moment logging statements are called directly from the logging
module (e.g. logging.info(...)
) - this implicitly creates a root logger and adds a default handler if it doesn't exist.
This can cause unwanted output for application developers.
The following links propose some best practice for logging in a library:
http://docs.python-guide.org/en/latest/writing/logging/
https://docs.python.org/3/howto/logging.html#configuring-logging-for-a-library
My understanding from the links above would be to make the following changes in betfairlightweight
:
betfairlightweight.__init__.py
:# Set default logging handler to avoid "No handler found" warnings.
import logging
try: # Python 2.7+
from logging import NullHandler
except ImportError:
class NullHandler(logging.Handler):
def emit(self, record):
pass
logging.getLogger(__name__).addHandler(NullHandler())
This would mean by default nothing would get logged by library unless configured by the user (this is optional since if the next point is implemented, it's easy for the user to configure the library to disable logging).
logging.*
add the following at the module level:logger = logging.getLogger(__name__)
And then switch all logging.*
statements to logger.*
statements.
I'm happy to submit a pull requests with the changes, but I wanted to check whether you had any issues with it or other ideas first.
betfairstream.py's _connect
sets self._running = True
before the socket connection actually succeeds.
If the socket fails to connect for some reason (see example stacktrace that occurs when the internet is not connected), betfairlightweight will think it is running. This causes issues in error handling because you can't trust the _running
attribute.
(PS sorry for filing another bug without resolving my others first!)
Example of socket connect failure when the internet is down:
File "/home/anon/.virtualenvs/anon-app/local/lib/python2.7/site-packages/betfairlightweight/streaming/betfairstream.py", line 194, in _send
self._connect()
File "/home/anon/.virtualenvs/anon-app/local/lib/python2.7/site-packages/betfairlightweight/streaming/betfairstream.py", line 131, in _connect
self._socket = self._create_socket()
File "/home/anon/.virtualenvs/anon-app/local/lib/python2.7/site-packages/betfairlightweight/streaming/betfairstream.py", line 139, in _create_socket
s.connect((self.__host, self.__port))
File "/usr/lib/python2.7/ssl.py", line 433, in connect
self._real_connect(addr, False)
File "/usr/lib/python2.7/ssl.py", line 420, in _real_connect
socket.connect(self, addr)
File "/usr/lib/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
gaierror: [Errno -3] Temporary failure in name resolution
Hi,
Thanks for your library.
Im facing some issues with streaming only, the rest working just fine ! (great work though)
I tried this
betfair_socket = trading.streaming.create_stream(unique_id=1) betfair_socket.start(async=False) betfair_socket.authenticate() betfair_socket.subscribe_to_markets(market_filter={'eventTypeIds':['1'], 'inPlayOnly': True, 'marketTypes':["MATCH_ODDS"]}, market_data_filter={'fields': ['EX_BEST_OFFERS']})
and I'm ending up with this result:
`1 {"op":"connection","connectionId":"006-051016081016-209520"}
None {"op":"status","statusCode":"FAILURE","errorCode":"TIMEOUT","errorMessage":"Connection is not subscribed and is idle: 15000 ms","connectionClosed":true,"connectionId":"006-051016081016-209520"}
`
Any help pls where im messing up ?
Once more thanks again for your effort.
Regards
I've run into the following error when trying to run an order stream:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/home/aaron/xxx/betfair_order_feed.py", line 94, in betfair_stream_run
betfair_stream_socket.start()
File "/home/aaron/tmp/venv/betfairlightweight/local/lib/python2.7/site-packages/betfairlightweight/streaming/betfairstream.py", line 49, in start
self._read_loop()
File "/home/aaron/tmp/venv/betfairlightweight/local/lib/python2.7/site-packages/betfairlightweight/streaming/betfairstream.py", line 177, in _read_loop
self._data(received_data)
File "/home/aaron/tmp/venv/betfairlightweight/local/lib/python2.7/site-packages/betfairlightweight/streaming/betfairstream.py", line 204, in _data
if self.listener.on_data(received_data) is False:
File "/home/aaron/tmp/venv/betfairlightweight/local/lib/python2.7/site-packages/betfairlightweight/streaming/listener.py", line 83, in on_data
self._on_change_message(data, unique_id)
File "/home/aaron/tmp/venv/betfairlightweight/local/lib/python2.7/site-packages/betfairlightweight/streaming/listener.py", line 108, in _on_change_message
self.stream.on_subscribe(data)
File "/home/aaron/tmp/venv/betfairlightweight/local/lib/python2.7/site-packages/betfairlightweight/streaming/stream.py", line 40, in on_subscribe
self._process(book_data, publish_time)
File "/home/aaron/tmp/venv/betfairlightweight/local/lib/python2.7/site-packages/betfairlightweight/streaming/stream.py", line 132, in _process
self._caches[market_id] = OrderBookCache(publish_time=publish_time, **order_book)
File "/home/aaron/tmp/venv/betfairlightweight/local/lib/python2.7/site-packages/betfairlightweight/resources/streamingresources.py", line 488, in __init__
self.runners = [OrderBookRunner(**i) for i in kwargs.get('orc', [])]
TypeError: __init__() got an unexpected keyword argument 'hc'
This is because of data like the following received from Betfair:
{
{
"id": "1.131975445",
"orc": [
{
"hc": -0.25,
"fullImage": true,
"id": 4917452,
"mb": [
[
1.27,
4.37
]
]
}
]
}
There's no documentation about this field in the API docs: http://docs.developer.betfair.com/docs/display/1smk3cen4v3lu3yomq5qye0ni/Exchange+Stream+API#ExchangeStreamAPI-OCM/OrderChangeMessage
I've opened a ticket with BDP.
PR incoming.
The BetfairStream class has a member _running
which is used to indicate if the stream is... running or not.
If you run a BetfairStream in async mode and you get a socket exception in _read_loop, there's no way to determine if the thread has died as the class doesn't store a reference to the thread it starts and _running
isn't updated in the case of the SocketError exceptions; checking _running
from an outside thread would indicate that it's still going.
I think the best way to solve this is just with a finally
block when _running
is set to False
. I'm not sure whether any socket cleanup needs to be done in case of an error? My brief googling didn't give me a definitive answer.
Raised PR #46
I have created a slack group to discuss any issues:
betfairlightweight.slack.com
A race condition can occur between _add_stream
and MarketStream's _process
if you subscribe on an open/running socket.
Subscribing to markets creates an entire new MarketStream object which has an empty cache. If then an mcm
arrives on the stream before the img
, there will be a KeyError in the cache because _process
can't update or initialise the cache unless it has received an img
update.
If the listener already has a market stream set when the user is re-subscribing to markets, the existing MarketStream object/cache could be re-used (validated by its unique_id perhaps). Alternatively you can stop, start the socket and subscribe again but that seems expensive.
Add atomic function to get current marketbook or orderbook.
listener.snap(market_id='1.1234567')
This could remove the dependency for an output queue (make optional) and allow the user to use the listener as a cache that can be requested when required.
Create stream 'on_update' function:
def on_update(self, market_books): if self.output_queue: self.output.queue.put(market_books)
Looking at the cancelOrders API call:
http://docs.developer.betfair.com/docs/display/1smk3cen4v3lu3yomq5qye0ni/cancelOrders
https://github.com/liampauling/betfairlightweight/blob/master/betfairlightweight/endpoints/betting.py#L303-L320
According to the Betfair documentation there are no required parameters - however the Betting.cancel_orders
method requires market_id
and instructions
as positional args.
I think the most intuitive way to do things is that positional args correspond to required parameters and kwargs correspond to parameters which are not required.
At the moment in this specific instance (cancel_orders
), the user is required to pass parameters which they do not want to use and it's not immediately obvious what they should set it these parameters to if they don't want them to be used.
The utils.clean_locals(...)
function will not include keys whose value is None so any parameters which are not required can just be defaulted to None.
To resolve the issue Betting.cancel_orders
needs to have market_id
and instructions
changed to kwargs defaulted to None.
I haven't looked at any of the other methods but there's the possibility there are other instances of this, so I think a full pass of the methods on the endpoints should performed, comparing the args / kwargs to the correspondng Betfair API required / not required parameters.
I'm happy to both look through the API calls and put the resulting changes into a PR.
I've run into an issue using the order stream which results in exceptions like the following (order_queue is a Queue which is the output_queue for a StreamListener):
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/home/aaron/workspace/xxx/xxx/order_feed/order_feed.py", line 35, in run
order_book = self.order_queue.get()
File "/usr/local/lib/python2.7/dist-packages/betfairlightweight/resources/baseresource.py", line 63, in json
return json.dumps(self._data)
File "/usr/lib/python2.7/json/__init__.py", line 244, in dumps
return _default_encoder.encode(obj)
File "/usr/lib/python2.7/json/encoder.py", line 207, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode
return _iterencode(o, 0)
File "/usr/lib/python2.7/json/encoder.py", line 184, in default
raise TypeError(repr(o) + " is not JSON serializable")
TypeError: datetime.datetime(2017, 2, 16, 11, 18, 59) is not JSON serializable
On further testing, this appeared to happen when the CurrentOrders returned from the OrderBookCache contained an UnmatchedOrder which had a matched_date field that was not None (it was a datetime.datetime object).
As you can see from the placedDate field a few lines down, that datetime.datetime object is converted to its string representation when being serialised:
The terminology as described in the documentation is quite confusing:
It seems like an UnmatchedOrder can contain a matched_date in the cases of partial matches.
I've created a pull request which contains the change I made locally to get it to work - you may have a nicer way of dealing with it.
Today I had some issues with the keepalive not returning json.
Of course this caused an exception where it could not decode the json.
Should unexpected, non-json responses be caught more tidily?
I wasn't able to debug whether the keepAlive succeeded despite not returning json nor whether this was just a temporary issue with betfair.
When the stream connects a connection-id is returned, this is currently stored in the class (not mentioned in the init), will be handy to log when there is an error to make any bug communications with betfair easier.
Any objections?
Its a pain to use compared to the scores endpoint because its used for the website, whereas scores just requires appKey to be authorised.
I've come across the following in my streaming orders:
Traceback (most recent call last):
File "/usr/lib64/python2.7/threading.py", line 804, in __bootstrap_inner
self.run()
File "/usr/lib64/python2.7/threading.py", line 757, in run
self.__target(*self.__args, **self.__kwargs)
File "/xxx/betfair_order_feed.py", line 79, in betfair_stream_run
betfair_stream_socket.start()
File "/xxx/local/lib/python2.7/site-packages/betfairlightweight/streaming/betfairstream.py", line 49, in start
self._read_loop()
File "/xxx/local/lib/python2.7/site-packages/betfairlightweight/streaming/betfairstream.py", line 177, in _read_loop
self._data(received_data)
File "/xxx/local/lib/python2.7/site-packages/betfairlightweight/streaming/betfairstream.py", line 204, in _data
if self.listener.on_data(received_data) is False:
File "/xxx/venv/local/lib/python2.7/site-packages/betfairlightweight/streaming/listener.py", line 84, in on_data
self._on_change_message(data, unique_id)
File "/xxx/local/lib/python2.7/site-packages/betfairlightweight/streaming/listener.py", line 115, in _on_change_message
self.stream.on_update(data)
File "/xxx/local/lib/python2.7/site-packages/betfairlightweight/streaming/stream.py", line 58, in on_update
self._process(book_data, publish_time)
File "/xxx/local/lib/python2.7/site-packages/betfairlightweight/streaming/stream.py", line 132, in _process
order_book_cache.update_cache(order_book, publish_time)
File "/xxx/local/lib/python2.7/site-packages/betfairlightweight/resources/streamingresources.py", line 506, in update_cache
runner.update_unmatched(order_changes.get('uo', []))
File "/xxx/local/lib/python2.7/site-packages/betfairlightweight/resources/streamingresources.py", line 476, in update_unmatched
self.unmatched_orders[n] = UnmatchedOrder(**unmatched_order)
TypeError: __init__() takes at least 18 arguments (17 given)
This is from the following JSON:
"uo":[
{
"status":"EC",
"md":1497288589000,
"pd":1497288587000,
"pt":"L",
"sr":0,
"sv":34.2,
"id":"95102351117",
"p":3,
"s":34.2,
"rfs":"xxx",
"sm":0,
"sl":0,
"sc":0,
"ot":"L",
"rfo":"xxx",
"side":"L"
}
]
The unmatched order is missing the "rac" and "rc" fields for regulator auth code and regulator code respectively. These are defined as positional args and are thus required when constructing the object. I suggest they simply be turned into kwargs so that it's handled if they're missing.
PR incoming.
Just profiled my framework when backtesting, when ordered by cumtime you can see that the creation of the base resource is slow. When streaming this is amplified because so many market books are created.
Implementing #57 should help.
Welcome any thoughts on how to speed this up.
ncalls tottime percall cumtime percall filename:lineno(function)
655587/1320 3.554 0.000 8.619 0.007 //resources/baseresource.py:19(__init__)
655587/1320 2.660 0.000 8.591 0.007 //resources/baseresource.py:52(set_attributes)
21608/1276 0.290 0.000 8.503 0.007 //resources/baseresource.py:34(set_sub_resources)
30599/1282 0.862 0.000 8.488 0.007 //resources/baseresource.py:45(<listcomp>)
I find certificate handling a bit confusing at the moment, since I would expect it to work like for requests. That is the certs
argument for APIClient
should be either a path to a .pem file, or a tuple with the paths to cert and key files.
Also, here you want to use ```os.path.join``, because if self.certs
doesn't end with `/`, then the generated path is wrong, but I was not sure about opening a PR just for that.
Found an issue where the following became present in MarketBook:
{'size': 0, 'price': 0}
Tried to recreate but I can't looks like it was due to receiving an update to delete a price when it was not present in the cache.
Given my experience trying to get betfairlightweight built as a conda package on conda-forge, I think it would be good to have tests run on Windows via appveyor:
Like other CI platforms, it's free for open source projects. This would help the project catch platform dependent errors during development.
In the betfair documentation, a RunnerCatalogue's metadata is not mandatory. For some sports/events, particularly non-horse racing, metadata is not returned. Betfair RunnerCatalog
However in bettingresources.py#L190, metadata is configured as mandatory.
Since release 1.1.2
, list_market_catalogue
method fails when it tries to make RunnerCatalogue resources as part of the MarketCatalogue resource.
In this traceback, debugging showed that there was no metadata in the runners returned by listMarketCatalogue
: {u'handicap': 0.0, u'runnerName': u'San Antonio Spurs', u'selectionId': 237471, u'sortPriority': 1}
Traceback:
Traceback (most recent call last):
File "anon-app.py", line 63, in <module>
jobs.update_market_subscription_list()
File "/home/anon/github/anon-app/jobs/job.py", line 185, in update_market_subscription_list
market_id_list = self.betfair_api.betting.list_market_catalogue(market_filter, market_projection=market_projection, max_results=100, lightweight=False)
File "/home/anon/.virtualenvs/anon-app/local/lib/python2.7/site-packages/betfairlightweight/endpoints/betting.py", line 158, in list_market_catalogue
return self.process_response(response, resources.MarketCatalogue, elapsed_time, lightweight)
File "/home/anon/.virtualenvs/anon-app/local/lib/python2.7/site-packages/betfairlightweight/endpoints/baseendpoint.py", line 100, in process_response
return [resource(elapsed_time=elapsed_time, **x) for x in result]
File "/home/anon/.virtualenvs/anon-app/local/lib/python2.7/site-packages/betfairlightweight/resources/bettingresources.py", line 222, in __init__
self.runners = [RunnerCatalogue(**i) for i in kwargs.get('runners', [])]
TypeError: __init__() takes exactly 6 arguments (5 given)
Streaming to be updated to use BaseResource along with refactoring. In play service to be implemented although the api constantly changes due to it not being private.
Investigate the use of a marketFilter data structure to make it easier to make requests, may not be implemented if it is found to be slow.
New placeOrder options now available, customerRefStrategy/timeInForce etc. Need to add to resources / test integration.
betfairlightweight-1.1.8
list_market_book() work fine, until market has no orders.
But with placed order it raise exception __init__() got an unexpected keyword argument 'matchesByStrategy'
This is because RunnerBook does not take a matchesByStrategy as keyword argument.
In preparing a conda recipe on conda-forge, I observed several tests failing on windows when appveyor tried to run the tests:
======================================================================
FAIL: test_get_app_key (tests.test_baseclient.BaseClientTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "c:\users\josh\documents\github\betfairlightweight\tests\test_baseclient.py", line 94, in tes
t_get_app_key
self.client.get_app_key()
AssertionError: AppKeyError not raised
======================================================================
FAIL: test_strip_datetime (tests.test_baseresource.BaseResourceInit)
----------------------------------------------------------------------
Traceback (most recent call last):
File "c:\users\josh\documents\github\betfairlightweight\tests\test_baseresource.py", line 87, in t
est_strip_datetime
assert type(stripped) == datetime.datetime
AssertionError
======================================================================
FAIL: test_strip_datetime_resource (tests.test_baseresource.BaseResourceInit)
----------------------------------------------------------------------
Traceback (most recent call last):
File "c:\users\josh\documents\github\betfairlightweight\tests\test_baseresource.py", line 116, in
test_strip_datetime_resource
assert model_response.datetime_int == datetime.datetime.utcfromtimestamp(1465631675000 / 1e3)
AssertionError
----------------------------------------------------------------------
Ran 203 tests in 0.844s
FAILED (failures=3)
test_get_app_key:
This test fails because os.environ.get(self.username)
, where self.username=="username"
returns a non-None response since USERNAME
is a standard windows environment variable (note that Windows env variables are case insensitive).
test_strip_datetime:
Fails on py27 since integer
is a long
rather than an int
. There is an easy fix for this.
test_strip_datetime_resource
There is a platform dependent exception that is raised so only catching ValueError
is insufficient.
Would it be possible to just pass a lightweight
flag to the APIClient rather than to every function? I think it's more likely people would use it for all/none of their API calls rather than for specific API calls.
Thoughts?
The following MarketCatalogue JSON was received from Betfair and caused the following error:
{
"marketStartTime": "2017-06-06T09:00:00.000Z",
"competition": {
"id": "11330922",
"name": "Prostejov Challenger 2017"
},
"marketId": "1.132045327",
"totalMatched": 17.06,
"marketName": "Match Odds",
"event": {
"timezone": "UTC",
"openDate": "2017-06-06T09:00:00.000Z",
"id": "28260245",
"countryCode": "CZ",
"name": "Marrero/Paes v Knowle/Zelenay"
},
"runners": [
{
"handicap": 0,
"runnerName": "Knowle/Zelenay",
"selectionId": 5717065,
"sortPriority": 1
},
{
"handicap": 0,
"runnerName": "Marrero/Paes.",
"selectionId": 8786145,
"sortPriority": 2
},
{
"handicap": 0,
"sortPriority": 3,
"selectionId": 8781561
}
]
}
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/home/aaron/workspace/sportsrisq.traderng/sportsrisq/traderng/feed/betfair_market_feed.py", line 42, in betfair_market_feed
locale=config.get('markets', {}).get('locale')
File "/home/aaron/tmp/venv/betfairlightweight/local/lib/python2.7/site-packages/betfairlightweight/endpoints/betting.py", line 158, in list_market_catalogue
return self.process_response(response, resources.MarketCatalogue, elapsed_time, lightweight)
File "/home/aaron/tmp/venv/betfairlightweight/local/lib/python2.7/site-packages/betfairlightweight/endpoints/baseendpoint.py", line 100, in process_response
return [resource(elapsed_time=elapsed_time, **x) for x in result]
File "/home/aaron/tmp/venv/betfairlightweight/local/lib/python2.7/site-packages/betfairlightweight/resources/bettingresources.py", line 231, in __init__
self.runners = [RunnerCatalogue(**i) for i in kwargs.get('runners', [])]
TypeError: __init__() takes at least 5 arguments (4 given)
The third runner is missing the runnerName field - which is listed a required field in the Betfair API docs: http://docs.developer.betfair.com/docs/display/1smk3cen4v3lu3yomq5qye0ni/Betting+Type+Definitions#BettingTypeDefinitions-RunnerCatalog
Because of this (and I think rightly so) the constructor for a RunnerCatalogue doesn't have a default value for runnerName: https://github.com/liampauling/betfairlightweight/blob/master/betfairlightweight/resources/bettingresources.py#L190-L195
I'm not really sure what the solution is here - maybe it's just to do nothing and accept that Betfair sends malformed data sometimes - however that will cause a listMarketCatalogue call to error in the above way. If all you want is the JSON then you can avoid it by using the lightweight
parameter in the API call.
I have raised a ticket with BDP but I imagine the response will be it was just a mistake.
The bigger question is whether resources should handle fields which are missing which are listed as required in the API specification? It's also a question which I don't have the answer to :)
market_ids does not get returned in the same way as in the betfair API
event_ids = [28200940, 28200941, 28200939, 28200937, 28202622, 28188158, 28202623, 28188159, 28202621, 28188157, 28202619, 28202616, 28202617, 28202614, 28202615, 28196677, 28202626, 28188160, 28202625, 28188176, 28188177]
markets = betfairclient.trading.betting.list_market_catalogue(
max_results=100,
filter=filters.market_filter(
event_ids=event_ids,
market_type_codes = ["MATCH_ODDS"]
))
yields mismatch between market_id and eventId
I looked briefly at the code but could not see any reordering, I could've missed something to. :)
I was getting this error:
self.output_queue.put(output_market_book)
AttributeError: 'NoneType' object has no attribute 'put'
To fix this in apiclient.py I added:
import queue
output_queue = queue.Queue()
and passed this to StreamListener in create_stream()
When recording data from a market I currently use .json() however this doesn't take advantage of streaming. The update or initial image is provided in .streaming_update but you then have to record data from the beginning of the session (not ideal if you only want in play data)
Therefore propose a new param called .streaming_image which creates a synthetic betfair image based on the current state of the market cache. The challenge will be limiting CPU as it will probably have to be calculated at the cache level for every update rather than a function in the MarketBook class.
Would you be interested in adding functionality like this (or does it exist and I am missing it again..)? https://github.com/limx0/btcmarkets#even-more-control
Basically, I have an optional arg that allows the api to simply build a dict of the request to be sent, but allows the user more control over which http library they use, rather than being tied to requests.
This is useful for async workflows, and particularly important when doing Inplay betting (don't want code to be handing while your order inserts)
Thoughts?
Use ciso8601 for parsing datetimes as its super quick.
Add thread lock to TransactionCount to prevent race conditions when using multiple threads.
When I call stop() on the listener socket I get the following AttributeError.
<snip>
File "/home/anon/.virtualenvs/anon/local/lib/python2.7/site-packages/betfairlightweight/streaming/betfairstream.py", line 56, in stop
if self._socket is not None and not self._socket._closed:
AttributeError: 'SSLSocket' object has no attribute '_closed'
I can't find this _closed
attribute in any of the Python documentation or else I'd try and file a PR for you!
It seems that on handicap markets the selectionId will be repeated:
[{"marketId":"1.131434037","isMarketDataDelayed":false,"status":"OPEN","betDelay":0,"bspReconciled":false,"complete":true,"inplay":false,"numberOfWinners":0,"numberOfRunners":66,"numberOfActiveRunners":66,"lastMatchTime":"2017-05-13T09:06:13.495Z","totalMatched":350.69,"totalAvailable":15649.09,"crossMatching":true,"runnersVoidable":false,"version":1645619435,"runners":[{"selectionId":579339,"handicap":-4.0,"status":"ACTIVE"},{"selectionId":197960,"handicap":4.0,"status":"ACTIVE"},{"selectionId":579339,"handicap":-3.75,"status":"ACTIVE"},{"selectionId":197960,"handicap":3.75,"status":"ACTIVE"},{"selectionId":579339,"handicap":-3.5,"status":"ACTIVE"},{"selectionId":197960,"handicap":3.5,"status":"ACTIVE"},{"selectionId":579339,"handicap":-3.25,"status":"ACTIVE"},{"selectionId":197960,"handicap":3.25,"status":"ACTIVE"}, ...
The streaming cache uses the selectionId as a primary key which isn't going to work on handicap markets.
The examples are broken, probably since commit e16e0d9
I've tried to fix it but was not successful. I'll try again if nobody else can get to it sooner.
Hi, @liampauling.
I think it's not a good idea to use snake_case in func/arg names.
Of course, it is accepted in the python, but the API-NG is done in camelCase mode. Continuous conversions are hardly reasonable. Also, this adds confusion to the code, for example, when creating an object, we use one style
cancel_bet = filters.cancel_instruction(bet_id=bet.id)
and accessing it's fields in a different style
foo(cancel_bet["betId"])
It's easy to make a typo that will be difficult to catch. Working with API-NG documentation you get used to camel style, etc. Therefore, I suggest using camelCase in the library.
What do you think?
If a socket timeout occurs in the _read_loop function then the socket is stopped and this affects services that are expecting data from the listener.
It's my opinion that the _read_loop should recover from the timeout and keep receiving data without manual intervention from the user.
except (socket.timeout, socket.error) as e:
self.stop()
raise SocketError('[Connect: %s]: Socket %s' % (self.unique_id, e))
Betfair has a rest API endpoint, although it doesn't follow any of the REST principles it does remove some of the complications lightweight solves. Not sure why I used the json-rpc endpoint in the first place.
Outcome will make the code easier to read and slightly quicker.
@synapticarbors regarding OpenSSL, this is a requirement of Requests, what errors are you getting on the conda build?
betfairlightweight was initially developed to be a wrapper that only abstracted the requests part of calling endpoints and required json as a parameter. However in order to make things simpler I would like to propose the replacing params with the correct parameters per endpoint.
def list_event_types(self, params=None, session=None):
def list_event_types(self, filter=MarketFilter(), locale=None, session=None, params=None):
:param filter: MarketFilter
:param locale: The language used for the response
:param session: Requests session object
:param params: Json request, default if not None
The added complication will be that more objects will have to be created/imported, for example the MarketFilter before a request can be made.
#34 provided by @rozzac90 is basically what I am referring to but with backwards compatibility.
I wanted to play around with this package but due to some constraints with other tooling, couldn't upgrade to Python 3. I spent a little bit of time making a few modifications such that the test suite passes under both Python 2.7 and 3.5. I'm happy to open a pull request, but wanted to know if you were open to possibly merging it or if you'd prefer to keep it Python 3 only for code purity (which I totally understand).
Currently if clk/initial_clk are provided BetfairStream will attempt a reconnect however listener.register_stream wipes all stream data. This will then cause issues because updates will be provided from betfair but the images have been removed.
If it is a reconnect the listener needs to just update the unique_id of the stream and not delete anything. This could maybe be the default and a new function created which 'refreshes' the listener if it wants to be reused.
If there is an error in the stream and connectionClosed is True the BetfairStream will quietly close, considering changing this to raise an error instead.
Add a raise error after this line so that the socket cleans itself up first.
https://github.com/liampauling/betfairlightweight/blob/master/betfairlightweight/streaming/betfairstream.py#L205
Betfair are adding a streaming status:
http://forum.bdp.betfair.com/showthread.php?t=3428
Probably worth adding the ability to choose the streaming endpoint as well.
This is not actually an issue, But there is no way to contact people on github. I am having issues with my app key not being authorized, Do you happen to know if this has to be a live key, And if it has to be specially activated by betfair?
Thanks,
Eyal.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.