No action at all (zero output) when evaluating a training result. about q-trader HOT 5 OPEN

I actually found out about this repo via Siraj's video which was the original I forked from. That said, I did issue a pull request to his. As stated above, you just need to issue a single buy which I do upon entry only and then allow the model to predict the rest of the way. I may look at performing a purchase when the local minimum has been reached in some window but currently I just use a bool for first iteration. I only trained up to 200 epochs, so I still have some more testing but it seems to be working descent. I will issue a pull request after some more testing is done, but should you wish to see changes, my repo is here: https://github.com/xtr33me/Reinforcement_Learning_for_Stock_Prediction

I also had to modify Sigmoid to allow for larger numbers and math.exp overflow issues. Unsure if this will help anyone, but got me moving forward again.

from q-trader.

I've found that the model isn't very stable. You can try evaluating the model with different saved instances like model_ep90, ep80 etc and there's some luck involved.

I think the "no action" occurs when the model determines that selling starts at and remains the optimal strategy, but it isn't able to sell because nothing has been bought. I'll see if I can find another way to implement this constraint.

You can also try changing the experience replay to random sampling by changing the first four lines of expReplay to mini_batch = random.sample(self.memory, batch_size).

from q-trader.

Hi applebull, did you just the exact code or do you have to modify a few things?

thanks.

from q-trader.

Hi Edward, I have the same problem and tried to use the above changes "mini_batch = random.sample(self.memory, batch_size)." but still got zero output.... Any idea? Thanks in advance.

from q-trader.

the same problem with no result!

python evaluate.py ^GSPC_2011 model_ep1000
/home/smilewater/anaconda3/envs/tensorflow-gpu/lib/python3.6/site-packages/h5py/init.py:34: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type.
from ._conv import register_converters as _register_converters
Using TensorFlow backend.
2018-07-22 21:54:45.515813: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2018-07-22 21:54:45.515834: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2018-07-22 21:54:45.515840: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2018-07-22 21:54:45.515844: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2018-07-22 21:54:45.515848: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2018-07-22 21:54:45.607514: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:893] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-07-22 21:54:45.607885: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 0 with properties:
name: GeForce GTX 1080 Ti
major: 6 minor: 1 memoryClockRate (GHz) 1.683
pciBusID 0000:02:00.0
Total memory: 10.91GiB
Free memory: 10.75GiB
2018-07-22 21:54:45.607912: I tensorflow/core/common_runtime/gpu/gpu_device.cc:976] DMA: 0
2018-07-22 21:54:45.607917: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 0: Y
2018-07-22 21:54:45.607924: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:02:00.0)

^GSPC_2011 Total Profit: $0.00

from q-trader.