chrism2671 / pytrendfollow Goto Github PK
View Code? Open in Web Editor NEWPyTrendFollow - systematic futures trading using trend following
License: MIT License
PyTrendFollow - systematic futures trading using trend following
License: MIT License
On march 26th 2007 VIX futures where rescaled as described in http://cfe.cboe.com/publish/CFEinfocirc/CFEIC07-003%20.pdf
Data imported from quandl before this date should be divided by 10 in order to backtest accurately.
Hi chrism2671,
I want to know how do you identify a trend from a sideway. If we use ema cross, an fake trend is very common.
Thanks
multiprocessing_on_dill.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\indexes\base.py", line 2525, in get_loc
return self._engine.get_loc(key)
File "pandas/_libs/index.pyx", line 117, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 139, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1265, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1273, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'open'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\site-packages\multiprocessing_on_dill\pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "C:\ProgramData\Anaconda3\lib\site-packages\multiprocessing_on_dill\pool.py", line 44, in mapstar
return list(map(*args))
File "C:/Code/PyTrendFollow-master\trading\accountcurve.py", line 61, in
self.memo_inst_calc = dict(pool.map(lambda x: (x.name, x.calculate()), self.portfolio))
File "C:/Code/PyTrendFollow-master\core\instrument.py", line 82, in calculate
rate = pd.Series(1, index=self.panama_prices().index)
File "C:/Code/PyTrendFollow-master\core\instrument.py", line 170, in panama_prices
return self.contracts()['close'].diff().to_frame().swaplevel().fillna(0).join(
File "C:/Code/PyTrendFollow-master\core\instrument.py", line 284, in contracts
data = data[(data['open'] > 0) & (data['volume'] > 1)]
File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\frame.py", line 2139, in getitem
return self._getitem_column(key)
File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\frame.py", line 2146, in _getitem_column
return self._get_item_cache(key)
File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\generic.py", line 1842, in _get_item_cache
values = self._data.get(item)
File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\internals.py", line 3843, in get
loc = self.items.get_loc(item)
File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\indexes\base.py", line 2527, in get_loc
return self._engine.get_loc(self._maybe_cast_indexer(key))
File "pandas/_libs/index.pyx", line 117, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 139, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1265, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1273, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'open'
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:/Code/PyTrendFollow-master/scheduler.py", line 111, in
main()
File "C:/Code/PyTrendFollow-master/scheduler.py", line 97, in main
schedule.run_pending()
File "C:\ProgramData\Anaconda3\lib\site-packages\schedule_init_.py", line 493, in run_pending
default_scheduler.run_pending()
File "C:\ProgramData\Anaconda3\lib\site-packages\schedule_init_.py", line 78, in run_pending
self.run_job(job)
File "C:\ProgramData\Anaconda3\lib\site-packages\schedule_init.py", line 131, in run_job
ret = job.run()
File "C:\ProgramData\Anaconda3\lib\site-packages\schedule_init.py", line 411, in run
ret = self.job_func()
File "C:/Code/PyTrendFollow-master/scheduler.py", line 51, in sync_trades
ib.sync_portfolio(p, acc=a, trade=trade)
File "C:/Code/PyTrendFollow-master\trading\ibstate.py", line 167, in sync_portfolio
frontier = portfolio.frontier(capital=acc.net)
File "C:/Code/PyTrendFollow-master\trading\portfolio.py", line 229, in frontier
c = self.curve(capital=capital)
File "C:/Code/PyTrendFollow-master\trading\portfolio.py", line 42, in curve
return accountCurve(list(self.valid_instruments().values()), **kw2)
File "C:/Code/PyTrendFollow-master\trading\accountcurve.py", line 35, in init
self.positions = self.instrument_positions()
File "C:/Code/PyTrendFollow-master\trading\accountcurve.py", line 71, in instrument_positions
self.memo_instrument_positions = pd.DataFrame({k: v['position'] for k, v in self.inst_calc().items()})
File "C:/Code/PyTrendFollow-master\trading\accountcurve.py", line 61, in inst_calc
self.memo_inst_calc = dict(pool.map(lambda x: (x.name, x.calculate()), self.portfolio))
File "C:\ProgramData\Anaconda3\lib\site-packages\multiprocessing_on_dill\pool.py", line 260, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "C:\ProgramData\Anaconda3\lib\site-packages\multiprocessing_on_dill\pool.py", line 608, in get
raise self._value
KeyError: 'open'
2018-03-13 04:22:49,332 - scheduler - ERROR - 'open'
multiprocessing_on_dill.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\indexes\base.py", line 2525, in get_loc
return self._engine.get_loc(key)
File "pandas/_libs/index.pyx", line 117, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 139, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1265, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1273, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'open'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\site-packages\multiprocessing_on_dill\pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "C:\ProgramData\Anaconda3\lib\site-packages\multiprocessing_on_dill\pool.py", line 44, in mapstar
return list(map(*args))
File "C:/Code/PyTrendFollow-master\trading\accountcurve.py", line 61, in
self.memo_inst_calc = dict(pool.map(lambda x: (x.name, x.calculate()), self.portfolio))
File "C:/Code/PyTrendFollow-master\core\instrument.py", line 82, in calculate
rate = pd.Series(1, index=self.panama_prices().index)
File "C:/Code/PyTrendFollow-master\core\instrument.py", line 170, in panama_prices
return self.contracts()['close'].diff().to_frame().swaplevel().fillna(0).join(
File "C:/Code/PyTrendFollow-master\core\instrument.py", line 284, in contracts
data = data[(data['open'] > 0) & (data['volume'] > 1)]
File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\frame.py", line 2139, in getitem
return self._getitem_column(key)
File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\frame.py", line 2146, in _getitem_column
return self._get_item_cache(key)
File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\generic.py", line 1842, in _get_item_cache
values = self._data.get(item)
File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\internals.py", line 3843, in get
loc = self.items.get_loc(item)
File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\indexes\base.py", line 2527, in get_loc
return self._engine.get_loc(self._maybe_cast_indexer(key))
File "pandas/_libs/index.pyx", line 117, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 139, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1265, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1273, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'open'
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:/Code/PyTrendFollow-master/scheduler.py", line 111, in
main()
File "C:/Code/PyTrendFollow-master/scheduler.py", line 97, in main
schedule.run_pending()
File "C:\ProgramData\Anaconda3\lib\site-packages\schedule_init_.py", line 493, in run_pending
default_scheduler.run_pending()
File "C:\ProgramData\Anaconda3\lib\site-packages\schedule_init_.py", line 78, in run_pending
self.run_job(job)
File "C:\ProgramData\Anaconda3\lib\site-packages\schedule_init.py", line 131, in run_job
ret = job.run()
File "C:\ProgramData\Anaconda3\lib\site-packages\schedule_init.py", line 411, in run
ret = self.job_func()
File "C:/Code/PyTrendFollow-master/scheduler.py", line 51, in sync_trades
ib.sync_portfolio(p, acc=a, trade=trade)
File "C:/Code/PyTrendFollow-master\trading\ibstate.py", line 167, in sync_portfolio
frontier = portfolio.frontier(capital=acc.net)
File "C:/Code/PyTrendFollow-master\trading\portfolio.py", line 229, in frontier
c = self.curve(capital=capital)
File "C:/Code/PyTrendFollow-master\trading\portfolio.py", line 42, in curve
return accountCurve(list(self.valid_instruments().values()), **kw2)
File "C:/Code/PyTrendFollow-master\trading\accountcurve.py", line 35, in init
self.positions = self.instrument_positions()
File "C:/Code/PyTrendFollow-master\trading\accountcurve.py", line 71, in instrument_positions
self.memo_instrument_positions = pd.DataFrame({k: v['position'] for k, v in self.inst_calc().items()})
File "C:/Code/PyTrendFollow-master\trading\accountcurve.py", line 61, in inst_calc
self.memo_inst_calc = dict(pool.map(lambda x: (x.name, x.calculate()), self.portfolio))
File "C:\ProgramData\Anaconda3\lib\site-packages\multiprocessing_on_dill\pool.py", line 260, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "C:\ProgramData\Anaconda3\lib\site-packages\multiprocessing_on_dill\pool.py", line 608, in get
raise self._value
KeyError: 'open'
Could not find thread pid_8636_id_202783656
Available: ['pid_8636_id_106792608', 'pid_8636_id_106791600', 'pid_8636_id_106792328', 'pid_8636_id_202818560', 'pid_8636_id_202741464', 'pid_8636_id_106792160', 'pid_8636_id_183354256', 'pid_8636_id_18849408']
Could not find thread pid_8636_id_202817776
Available: ['pid_8636_id_106792608', 'pid_8636_id_106791600', 'pid_8636_id_106792328', 'pid_8636_id_202818560', 'pid_8636_id_202741464', 'pid_8636_id_106792160', 'pid_8636_id_183354256', 'pid_8636_id_18849408']
Could not find thread pid_8636_id_106792720
Available: ['pid_8636_id_106792608', 'pid_8636_id_106791600', 'pid_8636_id_106792328', 'pid_8636_id_202818560', 'pid_8636_id_202741464', 'pid_8636_id_106792160', 'pid_8636_id_183354256', 'pid_8636_id_18849408']
Process finished with exit code 0
Best practice would be to use argparse for command line parameters.
One needs the appropriate version of libpython-dev installed in order to successfully install arch dependency. Make a note in the documents.
Currently core.instrument.Instrument
has a method panama_prices()
which computes the panama_prices going forward.
Feature: create a new function, or a boolean argument passed into panama_prices()
, which computes the panama_prices backwards (i.e. starting from the most recent date and going back).
On scheduling and after downloading data,
File "C:/Code/PyTrendFollow-master\trading\rules.py", line 54, in carry_spot
f = inst.spot() - inst.market_price().reset_index('contract', drop=True)
File "C:/Code/PyTrendFollow-master\core\spot.py", line 36, in get
raise Exception("No price data for symbol: %s" % self)
Exception: No price data for symbol: eurostoxx (spot)
i['cocoa'].contract_format(199309)
returns
AttributeError Traceback (most recent call last)
<ipython-input-78-d8ecb233dd15> in <module>()
----> 1 i['cocoa'].contract_format(199309)
~/PyTrendFollow/core/instrument.py in contract_format(self, contract)
236 """
237 year, month = contract_to_tuple(contract)
--> 238 if self.contract_name_format == 'cbot':
239 return self.quandl_symbol + cbot_month_code(month) + str(year)
240 else:
AttributeError: 'Instrument' object has no attribute 'contract_name_format'```
Update Installation instructions documentation for Mac
Hi everyone,
I tried changing the base currency to EUR but keep getting an error that prices are not found for USDEUR.
I simply added a dictionary with codename USDEUR and then changed the lambda x:x to lambda x: 1/x. as said in the guide. Ideally the code could figure out concurrent currencies by triangular multiplication or inverting.
it just shows that i have connected to the client, but just timeout when i downloaded
the data from ib it just warned me that it times out , then the graphical interfaces of my client shows
the status of history_data_center is inactive, i just want to use ib-api, but it never works, so
i really need your help to download the data from ib-api, Looking forward to hearing from you
It will be cleaner to move sample strategy implementation to it's own dir called TradeStrategy.
Hi Guys sorry couldn't find another way of reaching out to you so feel free to delete issue as see fit.
Basically I came here from RobCarvers issues list. I've read his systematic trader book a million times and largely get the whole concept.. Well I think I do until I try and use his pysystemtrade framework and my head explodes.
I'm a long time python+linux dev but frankly I have a hard time following his implementation and its made worse by the fact I'm not trading futures atm..( have reasons)
Just then I spot PyTrendFollow... yey!!
So I thought I'd have a dig around and see if I can add support for my CFD/FX broker. http://developer.oanda.com/rest-live-v20/introduction/
So 2 actual questions
Either way.. just wanted to say thanks, love the project and I can get my head round it so thats a good sign lol
cheers
Follow standard approach of requirements.txt in top-level directory. Remove need to run scripts/install_requirements.sh
Hi I got this error in running for corn price panama_pricec() fuction:
'
TypeError: Level type mismatch: 199312
'
Could i please get some help, whether this is related to the data level is only monthly frequency? is there any place i could change the config frequency for price regression functions? thanks :)
Hi there , Im having a bug when connecting to the IB api.
it's relating to the multi IB accounts
More specifically I get an error in the method _account_summary_handler of ibstate
acc = self.accounts[msg.account] KeyError: 'DU15195'
The problem is that I never see accounts initialised anywhere.
It is therefore trying to slice an empty dictionary.
Should the fix not be ?
acc=msg.account
Kind regards
Best to isolate all dependancies in a virtual environment, especially given the need for Python 3. Describe how to set up virtualenv in the docs.
Resolve issues arising from a quick check with pyflakes:
(PyTrendFollow) paul@paul-Satellite-R630:~/github/PyTrendFollow$ pyflakes config/
(PyTrendFollow) paul@paul-Satellite-R630:~/github/PyTrendFollow$ pyflakes core/
core/basestore.py:9: 'read_contract' imported but unused
core/basestore.py:9: 'drop_symbol' imported but unused
core/basestore.py:9: 'write_data' imported but unused
core/basestore.py:9: 'read_symbol' imported but unused
(PyTrendFollow) paul@paul-Satellite-R630:~/github/PyTrendFollow$ pyflakes data/
(PyTrendFollow) paul@paul-Satellite-R630:~/github/PyTrendFollow$ pyflakes tests/
tests/test_portfolio.py:1: 'trading' imported but unused
(PyTrendFollow) paul@paul-Satellite-R630:~/github/PyTrendFollow$ pyflakes trading/
trading/ibstate.py:314: undefined name 'ConnectionResetError'
trading/bootstrap_portfolio.py:2: 'np' imported but unused
trading/bootstrap_portfolio.py:7: 'sharpe' imported but unused
trading/accountcurve.py:10: 'os' imported but unused
(PyTrendFollow) paul@paul-Satellite-R630:~/github/PyTrendFollow$ pyflakes *.py
(PyTrendFollow) paul@paul-Satellite-R630:~/github/PyTrendFollow$
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.