samre12 / deep-trading-agent Goto Github PK

Traceback (most recent call last):
File "/home/arunavo/PycharmProjects/samre12Trading/trader/main.py", line 47, in
main("/home/arunavo/PycharmProjects/samre12Trading/trader/agent/config/config.cfg")
File "/home/arunavo/PycharmProjects/samre12Trading/trader/main.py", line 37, in main
agent.train()
File "/home/arunavo/PycharmProjects/samre12Trading/trader/agent/model/train.py", line 69, in train
self.replay_memory.set_history(state, supplementary)
File "/home/arunavo/PycharmProjects/samre12Trading/trader/agent/model/replay_memory.py", line 107, in set_history
self.add(state[length - 1], 0.0, 0, False, supp)
File "/home/arunavo/PycharmProjects/samre12Trading/trader/agent/model/replay_memory.py", line 53, in add
raise ValueError(INVALID_TIMESTEP)
ValueError: INVALID_TIMESTEP
ERROR:deep_trading_agent:Invalid Timestep with shapes (4,), (1,)

Process finished with exit code 1

/// In file : train.py
def train(self):
............

    state, supplementary = self.env.reset()
    self.history.set_history(state)
    self.replay_memory.set_history(state, supplementary)

#env.reset()
//returns -->> return self.historical_prices[self.current - self.history_length:self.current], np.array([1.0])

from file: cryptoenv.py
//its an np.array.shape --> 1
whereas supplementary needs 27 according to the config file

P.S: for branch adaptive_normalization

Hi, cool project.

Getting a weird error during training when using the same config as example.cfg:

KeyError: KeyError(<weakref at 0x7f3801e2a628; to 'tqdm' at 0x7f38019d88d0>,) in
 <bound method tqdm.__del__ of   0%|                    | 24866/25000000 [00:05<
1:24:55, 4901.45it/s]> ignored
Traceback (most recent call last):
  File "main.py", line 38, in <module>
    main(vars(args)['file_path'])
  File "main.py", line 29, in main
    agent.train()
  File "/home/ted/projects/deep-trading-agent/code/model/agent.py", line 71, in 
train
    self.observe(screen, reward, action, terminal, trade_rem)
  File "/home/ted/projects/deep-trading-agent/code/model/agent.py", line 170, in
 observe
    self.q_learning_mini_batch()
  File "/home/ted/projects/deep-trading-agent/code/model/agent.py", line 186, in
 q_learning_mini_batch
    self.t_trade_rem_t: trade_rem_t_plus_1
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.
py", line 905, in run
    run_metadata_ptr)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.
py", line 1137, in _run
    feed_dict_tensor, options, run_metadata)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.
py", line 1355, in _do_run
    options, run_metadata)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.
py", line 1374, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Input to reshape i
s a tensor with 80640 values, but the requested shape has 115200
         [[Node: t_q_network/Reshape_1 = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _d
evice="/job:localhost/replica:0/task:0/device:GPU:0"](t_q_network/conv_layers/co
nv_layer_2/conv_2/Relu, t_q_network/Reshape_1/shape)]]
         [[Node: t_q_network/q_values/BiasAdd/_139 = _Recv[client_terminated=fal
se, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/jo
b:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_na
me="edge_274_t_q_network/q_values/BiasAdd", tensor_type=DT_FLOAT, _device="/job:
localhost/replica:0/task:0/device:CPU:0"]()]]

Caused by op u't_q_network/Reshape_1', defined at:
  File "main.py", line 38, in <module>
    main(vars(args)['file_path'])
  File "main.py", line 28, in main
    agent = Agent(sess, logger, config, env)
  File "/home/ted/projects/deep-trading-agent/code/model/agent.py", line 44, in 
__init__
    self.build_dqn(params)
  File "/home/ted/projects/deep-trading-agent/code/model/agent.py", line 254, in
 build_dqn
    self.t_q.build_model((self.t_s_t, self.t_trade_rem_t))
  File "/home/ted/projects/deep-trading-agent/code/model/deepsense.py", line 150
, in build_model
    self.params.window_size * self.params.filter_sizes[-1]
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_array_o
ps.py", line 3903, in reshape
    "Reshape", tensor=tensor, shape=shape, name=name)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_de
f_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.p
y", line 3271, in create_op
    op_def=op_def)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.p
y", line 1650, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-
access

InvalidArgumentError (see above for traceback): Input to reshape is a tensor wit
h 80640 values, but the requested shape has 115200
         [[Node: t_q_network/Reshape_1 = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _d
evice="/job:localhost/replica:0/task:0/device:GPU:0"](t_q_network/conv_layers/co
nv_layer_2/conv_2/Relu, t_q_network/Reshape_1/shape)]]
         [[Node: t_q_network/q_values/BiasAdd/_139 = _Recv[client_terminated=fal
se, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/jo
b:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_na
me="edge_274_t_q_network/q_values/BiasAdd", tensor_type=DT_FLOAT, _device="/job:
localhost/replica:0/task:0/device:CPU:0"]()]]

Any idea for what this is from?

Hello,
I am running main.py and got the following error:

Traceback (most recent call last):
  File "main.py", line 36, in <module>
    main(vars(args)['file_path'])
  File "main.py", line 27, in main
    agent.train()
  File "deep-trading-agent/code/model/agent.py", line 70, in train
    screen, reward, terminal = self.env.act(action)
  File "deep-trading-agent/code/model/environment.py", line 62, in act
    if self.action_dict[action] is LONG:
TypeError: unhashable type: 'numpy.ndarray'

Could you tell me what goes wrong?
Thanks

Hi,
I tried using another datasets, one with a forex pair and another one with the BTC/USD pair but it gives me that error for every dataset. With the original dataset (btc.csv) this error doesn't happen, and the other ones has the same exact column names and data types. The only thing I did was changing the config file to take this new dataset.

Here's a look of my custom BTC dataset.

Also, the people on the internet say that I should decrease the learning rate, so I did, but the problem persist. I also tried messing around with the parameters from model/baseagent.py but same.
I ask here because I tried everything, including changing the order of the colums in the dataset.

Thank you!

0%|                   | 49900/100000000 [00:15<8:40:07, 3202.78it/s]Traceback (most recent call last):
  File "main.py", line 38, in <module>
    main(vars(args)['file_path'])
  File "main.py", line 29, in main
    agent.train()
  File "/deep-trading-agent/code/model/agent.py", line 71, in train
    self.observe(screen, reward, action, terminal, trade_rem)
  File "/deep-trading-agent/code/model/agent.py", line 170, in observe
    self.q_learning_mini_batch()
  File "/deep-trading-agent/code/model/agent.py", line 204, in q_learning_mini_batch
    self.learning_rate_step: self.step
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 778, in run
    run_metadata_ptr)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 982, in _run
    feed_dict_string, options, run_metadata)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1032, in _do_run
    target_list, options, run_metadata)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1052, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Nan in summary histogram for: q_network/avg_q_summary/q/2
         [[Node: q_network/avg_q_summary/q/2 = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](q_network/avg_q_summary/q/2/tag, q_network/avg_q_summary/strided_slice_2)]]

Caused by op u'q_network/avg_q_summary/q/2', defined at:
  File "main.py", line 38, in <module>
    main(vars(args)['file_path'])
  File "main.py", line 28, in main
    agent = Agent(sess, logger, config, env)
  File "/deep-trading-agent/code/model/agent.py", line 44, in __init__
    self.build_dqn(params)
  File "/deep-trading-agent/code/model/agent.py", line 237, in build_dqn
    self.q.build_model((self.s_t, self.trade_rem_t))
  File "/deep-trading-agent/code/model/deepsense.py", line 205, in build_model
    self._avg_q_summary.append(tf.summary.histogram('q/{}'.format(idx), avg_q[idx]))
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/summary/summary.py", line 209, in histogram
    tag=scope.rstrip('/'), values=values, name=scope)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_logging_ops.py", line 139, in _histogram_summary
    name=name)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 768, in apply_op
    op_def=op_def)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2336, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1228, in __init__
    self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): Nan in summary histogram for: q_network/avg_q_summary/q/2
         [[Node: q_network/avg_q_summary/q/2 = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](q_network/avg_q_summary/q/2/tag, q_network/avg_q_summary/strided_slice_2)]]

Hello, I tried running the code with the config file shown in the image and I still getting that error. But the 'global' section is at the beginning of the file.

>sudo cat config/config.cfg
[global]
PARENT_DIR = /home/mmnkl9/deep-trading-agent/code/

[logging]
LOG_FILE = logs/run.log
SAVE_DIR = logs/saved_models/
TENSORBOARD_LOG_DIR = logs/tensorboard/

[preprocessing]
DATASET_PATH = data/btc.csv

[dataset]
BATCH_SIZE = 32
HISTORY_LENGTH = 180

Do you have any plan to add SMI, RSI, ATR indices to the input state?

Hi, I have some problems...
the first is in utils/util

def get_config_parser(filename): config = ConfigParser(allow_no_value=True) config.read(filename) return config

filename is a NoneType object, I don't udenderstand wich file i must put.

I tried with "config" instead "filename" but i receive this error:

`Traceback (most recent call last):
File "C:\Users\vadil\Anaconda3\envs\Test5\lib\configparser.py", line 1138, in _unify_values
sectiondict = self._sections[section]
KeyError: 'global'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:/Users/vadil/PycharmProjects/Test5/main.py", line 38, in
main(vars(args)['file_path'])
File "C:/Users/vadil/PycharmProjects/Test5/main.py", line 21, in main
config = get_config(config_parser)
File "C:\Users\vadil\PycharmProjects\Test5\utils\config.py", line 17, in get_config
config[PARENT_DIR] = config_parser.get(GLOBAL, PARENT_DIR)
File "C:\Users\vadil\Anaconda3\envs\Test5\lib\configparser.py", line 781, in get
d = self._unify_values(section, vars)
File "C:\Users\vadil\Anaconda3\envs\Test5\lib\configparser.py", line 1141, in _unify_values
raise NoSectionError(section)
configparser.NoSectionError: No section: 'global'

Process finished with exit code 1`

I love this project, in my university no one teach RL than I started this study alone.

Hi I am having difficulty in understanding the reward function. More specifically I am not able to understand what does "amount of long currency" and "amount of short currency" mean?

I would appreciate if you could elaborate a on the reward function and the terms used in it.

Thanks.

Problems with Python 3.6 - Unable to get past this point

2018-05-21 08:58:16.038248: I T:\src\github\tensorflow\tensorflow\core\platform\cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 2018-05-21 08:58:16.278040: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:1356] Found device 0 with properties: name: GeForce GTX 1050 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.4425 pciBusID: 0000:02:00.0 totalMemory: 4.00GiB freeMemory: 3.29GiB 2018-05-21 08:58:16.278375: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:1435] Adding visible gpu devices: 0 2018-05-21 08:58:16.779930: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix: 2018-05-21 08:58:16.780127: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:929] 0 2018-05-21 08:58:16.780253: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:942] 0: N 2018-05-21 08:58:16.780467: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3025 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1050 Ti, pci bus id: 0000:02:00.0, compute capability: 6.1) INFO: Making new env: UnRealizedPnLEnv-v0 Traceback (most recent call last): File "D:/deep-trading-agent-dev/code/main.py", line 44, in <module> main(vars(args)['file_path']) File "D:/deep-trading-agent-dev/code/main.py", line 34, in main agent = Agent(sess, logger, config, env) File "D:\deep-trading-agent-dev\code\model\agent.py", line 252, in __init__ self.build_dqn(params) File "D:\deep-trading-agent-dev\code\model\agent.py", line 448, in build_dqn print (self.q_gru_keep_prob) AttributeError: 'Agent' object has no attribute 'q_gru_keep_prob'

I am a very beginner in ML and so have little understanding on what this model capable of.
Only want to suggest and, hopefully, get a feedback.

If we analyze only raw exchange data, as trades in order books and closed ones, we may have:

• current buy and sell price
• buy and sell volumes in some period of time
• number of buy and sell orders closed (they are not always correlated with volume)
• buy and sell order book (for example, volume needed to move price for particular percent, or measure activity in order book)

As post-processing of these values, we can divide them by SMA, measure speed they are changing.

Hi!
Have a question about training.

After 16 hours of training, I still get average reward 0.
Will be happy if you can explain what can be wrong?
Maybe it's a problem with default setup parameters?

25%|███▊ | 6374997/25000000 [16:18:15<48:41:33, 106.25it/s]INFO:deep_trading_agent:avg_r: 0.0000, avg_l: 0.000135, avg_q: -0.001807, avg_ep_r: 0.0000, max_ep_r: 0.0911, min_ep_r: -0.0984, # game: 5000
26%|███▊ | 6399993/25000000 [16:22:11<48:18:45, 106.94it/s]INFO:deep_trading_agent:avg_r: 0.0000, avg_l: 0.000135, avg_q: -0.001469, avg_ep_r: 0.0000, max_ep_r: 0.0985, min_ep_r: -0.0641, # game: 5000
26%|███▊ | 6424989/25000000 [16:26:07<48:06:53, 107.24it/s]INFO:deep_trading_agent:avg_r: 0.0000, avg_l: 0.000138, avg_q: -0.001775, avg_ep_r: 0.0001, max_ep_r: 0.1445, min_ep_r: -0.0460, # game: 5000
26%|███▊ | 6449993/25000000 [16:30:03<48:41:25, 105.83it/s]INFO:deep_trading_agent:avg_r: -0.0000, avg_l: 0.000134, avg_q: -0.001525, avg_ep_r: -0.0000, max_ep_r: 0.0223, min_ep_r: -0.0371, # game: 5000
26%|███▉ | 6477033/25000000 [16:34:16<47:10:48, 109.06it/s]INFO:deep_trading_agent:avg_r: -0.0000, avg_l: 0.000138, avg_q: -0.002763, avg_ep_r: -0.0000, max_ep_r: 0.0302, min_ep_r: -0.0762, # game: 5000
26%|███▉ | 6499197/25000000 [16:37:41<47:10:50, 108.92it/s]INFO:deep_trading_agent:avg_r: 0.0000, avg_l: 0.000142, avg_q: -0.003163, avg_ep_r: 0.0000, max_ep_r: 0.0352, min_ep_r: -0.0225, # game: 5000
26%|███▉ | 6526765/25000000 [16:41:56<47:30:54, 108.00it/s]INFO:deep_trading_agent:avg_r: -0.0000, avg_l: 0.000135, avg_q: -0.003114, avg_ep_r: -0.0000, max_ep_r: 0.0253, min_ep_r: -0.1445, # game: 5000
26%|███▉ | 6551381/25000000 [16:45:43<47:47:03, 107.25it/s]INFO:deep_trading_agent:avg_r: 0.0000, avg_l: 0.000131, avg_q: -0.002506, avg_ep_r: 0.0000, max_ep_r: 0.0643, min_ep_r: -0.0199, # game: 5000
26%|███▉ | 6577145/25000000 [16:49:41<47:26:52, 107.85it/s]INFO:deep_trading_agent:avg_r: -0.0000, avg_l: 0.000137, avg_q: -0.001795, avg_ep_r: -0.0000, max_ep_r: 0.0300, min_ep_r: -0.1185, # game: 5000
26%|███▉ | 6599989/25000000 [16:53:14<46:38:00, 109.60it/s]INFO:deep_trading_agent:avg_r: -0.0000, avg_l: 0.000137, avg_q: -0.002334, avg_ep_r: -0.0000, max_ep_r: 0.0495, min_ep_r: -0.1122, # game: 5000

Hi @samre12 ,

It's looks cool your work.
Can you share example of the config file with us?

I am the only one who have computation performance issue?When i run the script the requested computation time is about 250 hours...

https://gist.github.com/johndpope/b8a1f334e1d1769e73391ad6270bbff9

for easier config editing.

Why On Earth would you use python 2.7 ? that completely makes no sense but anyway good project

Hi! Tried to run your agent but found an error - "ModuleNotFoundError: No module named 'ConfigParser'" - seems that this file was not committed
ah, okay - it's python 2.7

it seems without a sense of risk / reward or position sizing - the trained model is a glorified heads + tails flip calculator based off previous data.

it maybe beyond scope of project - but to build an autonomous agent that can a variety of actions - need to somehow incorporate this to win the trading game. Perhaps there's two trained models.

https://github.com/m-esgana/poor-trader-py/blob/097fbb10c217abe0db5a2cd55d73e0ad2990acd9/poor_trader/trading.py

Hi samre12,

After the training using the main.py, how can I test on another set of data? Thanks.

Thanks,
Lam

For kaggle's btceUSD dataset we got these last 5 lines in which BTC(USD) weighted price =2130~2139:

1496188560,2130,2130,2129.999,2129.999,2.35330472,5012.53827,2129.999667
1496188620,2130,2134.8,2130,2134.8,12.03238662,25648.897767,2131.6550554
1496188680,2133.2,2134.8,2133.2,2134.8,1.40799439,3005.4756141,2134.5792536
1496188740,2134.8,2143.79,2134.8,2134.801,3.19062001,6824.2637131,2138.8519133
1496188800,2139.999,2140,2134.35,2139.999,6.73707917,14413.324651,2139.4025938

When I make a request to cryptocompare website for BTC(USD) I get this where the price suddenly jumps to 7656.98 USD at the same timestamp:

{"RAW":{"BTC":{"USD":{"TYPE":"5","MARKET":"CCCAGG","FROMSYMBOL":"BTC","TOSYMBOL":"USD","FLAGS":"1","PRICE":7656.98,"LASTUPDATE":1511013600,"LASTVOLUME":0.26175057,"LASTVOLUMETO":2002.3918605000001,"LASTTRADEID":"26754557","VOLUMEDAY":48775.74066467763,"VOLUMEDAYTO":372532531.85840243,"VOLUME24HOUR":89903.71924385922,"VOLUME24HOURTO":693640577.3317167,"OPENDAY":7699.95,"HIGHDAY":7798.12,"LOWDAY":7458.9,"OPEN24HOUR":7829.71,"HIGH24HOUR":7982.5,"LOW24HOUR":7446.1,"LASTMARKET":"Bitstamp","CHANGE24HOUR":-172.73000000000047,"CHANGEPCT24HOUR":-2.2060842611029075,"CHANGEDAY":-42.970000000000255,"CHANGEPCTDAY":-0.5580555717894304,"SUPPLY":16686375,"MKTCAP":127767239647.5,"TOTALVOLUME24H":480649.677315126,"TOTALVOLUME24HTO":3685574563.3642445}}}

What am I missing here sir?