Aim of this repository is solving ml4trade gym environment using reinforcement learning algorithms (specifically A2C and PPO).
- python 3.7 - 3.10
- pip requirements
- data to process by the environment
The data used by ml4trade by default can be retrieved via provided bash script download_all.sh. This is described in Installation step.
Pip-install requirements:
pip install -r requirements.txt
Download default required data (~1.5GB in total):
bash data/download_all.sh
Necessary files will be downloaded, unpacked and stored in
automatically created .data/
directory.
You can run tests stored in test/
directory to see if everything works
as expected:
python -m unittest discover test
Start by using run.py
script. Arguments are handled by
Hydra framework: by default arguments are
taken from conf/ directory. They are proven to give good results, but
you can override them if needed. Seed can be optionally provided by
+run.seed=your_seed_here
, otherwise it is acquired from current datetime.
For example, given you want to run A2C
algorithm for 1,000,000
time steps
with n_steps
parameter set to 50
and seed
set to 42
:
python run.py agent=a2c run.train_steps=1e6 agent.n_steps=50 +run.seed=42
All parameters stored in conf/ are configurable.
Other things worth mentioning:
- results are stored in
outputs/
directory, during each run Hydra creates there a new directory where logs and artifacts are stored - artifacts mentioned previously include: saved best model according to the performance on the test env, history of the actions on the test env, various logs, performance plots
- consider using TensorBoard, TensorBoard logs are written during runs
run.py
was meant to be flexible and frequently changed according to needs.
Feel free to tinker with it.