henryzhongsc / half_tael_dqn Goto Github PK

Develop a set of evaluation functions to evaluate the status (weight) of certain metrics introduced in Anita folder.

Such functions should take _time, _currency_pair, _volume, trade_action (long, short, hold) and any other necessary parameters (e.g. historical trade_log and market info from oanda_interface.py) as inputs and return a float number between -1 to 1 as a reflection of the status of such metrics at this particular position.

closed due to contributor exited the program, task will be handled

• Manual needed for train_interface.py.
• Documentation needed for RL_brain.py, FX_env.py, and model_interface.py.

Influence the reward design of half_tael_v4.0.

Reference:

• Reward Shaping:
  • Using Natural Language for Reward Shaping in Reinforcement Learning | P. Goyal, S. Niekum, R. J. Mooney

• logs (tensorboard, trade_logs).
• CSV files (raw_data, arena_data).

• CHANGELOG.md

  • Till Scrum #11
  • Development journals.

• README.md

  • Anita information.
  • Run section instruction fix.
  • Right of ownership with Graphen.

• 🔰 User Manual for trade_interface.py

  • Untrack oanda_config.py.
  • New sample output.
  • Document CSV output parameters.

• 🔰 User Manual for anita_interface.py

  • Polish.
  • Hyperlink to master files.

Separate as:

• model_interface.py: for the main loop and FX, DQN, and Trade_Interface instance building.
• train_interace.py: a clean entry point of the program.
• model_config.py: configuration for run_model(), FX, DQN, Trade_Interface input.
• oanda_config.py (originally config.py): configuration for Oanda API connection.

Base on the Anita Folder.

Generate documentation to explain the design of trial_DQN v1.0 for future refactor to compliant with virtual trading platform v1.0.

cost_log.txt

• More versatile cost_graph output.
• Mark trade stamps on time-price graph.

Ref.

• Visualizing an Agent’s Thoughts and Actions |
  Arthur Juliani

To list out the future direction(s) of this project.

• Improving CNN:

  • More layers.
  • Tuning parameters regarding strides, filter, etc.

• Different model before DQN:

  • RNN.
  • LSTM (essentially DRQN).
  • Rainbow.

• Improve Anita:

  • Make another psy metrics on personalty which actually suits the role of reward indicator.
  • Apply reward shaping.
  • Dynamic weight adjustment between traits and persona-avatar.
    • Supervised learning?
    • NN for dynamic categorization rather than hardcored weight?

• Improve OANDA communication:

  • Submit trade request.
  • Read real time data.

• Deploy:

  • Train with local Linux server.
  • Train on cloud.

Reference

• RichardLitt/standard-readme.
• matiassingers/awesome-readme

• Reference
  * reinforcement-learning/DQN
  * https://www.datahubbs.com/deepmind-dqn/
  * Using Keras and Deep Q-Network to Play FlappyBird | Ben Lau
  * Beat Atari with Deep Reinforcement Learning! (Part 1: DQN)
  * floodsung/DQN-Atari-Tensorflow

Develop a set of evaluation functions to evaluate the status (weight) of certain metrics introduced in Anita folder.

Such functions should take _time, _currency_pair, _volume, trade_action (long, short, hold) and any other necessary parameters (e.g. historical trade_log) as inputs and return a float number between -1 to 1 as a reflection of the status of such metrics at this particular position.

• Reinforcement Learning for FX trading | Dai, Wang, Wang, Xu | Stanford U.
  • Paper
  • Slide
  • GitHub

• Update rebase pull request guideline with example.
• Add branch checkout comment guideline with example.
• Add usage guideline for pin with example.

Mostly regarding the 3 * 3200 output in CNN with 889 rows of trial data (300 rows for train and 300 rows for test for fast debug). To reproduce:

    _account_name = 'DQN_v3.0_trial_data_899_line'
    _currency_balance = {'USD': 0, 'EUR': 100000, 'GBP': 0}
    _from = "2018-04-13T18:30:00Z"
    # _to = "2018-04-19T03:27:00Z" # 4800 rows.
    _to = "2018-04-16T09:30:00Z" # 904 rows
    _interval = "M1"
    TI_train = TI.Trade_Interface(_account_name, _currency_balance, _from, _to, _interval)

• Supports trading among multiple (>2) currencies.
• Implemented with a model design which understands the relation between timeframes.
• Takes following sample data as input:
  * Train: USD_JPY_GBP_2017-01-01T00:00:00_2018-01-01T00:00:00_M5.csv (74,277 rows).
  * Test: USD_JPY_GBP_2018-01-01T00:00:00_2019-01-01T00:00:00_M5.csv (74,590 rows).
  * Lightweight trial data for development: USD_JPY_GBP_2019-01-01T00:00:00_2019-02-01T00:00:00_M5.csv (6,358 rows).

Reference:

• anita_v1.1 folder

Understanding the fundamental of DQN.

• 强化学习 Reinforcement Learning 教程系列l | Morvan Zhou
• DQN从入门到放弃5 深度解读DQN算法 | Flood Sung​
• RL — DQN Deep Q-network | Jonathan Hui
• 神经网络，BP算法的理解与推导 | 折射