burntt / finrl_crypto Goto Github PK

When i use :
python 1_optimize_cpcv.py

i get following error:
python 1_optimize_cpcv.py

python 1_optimize_cpcv.py
10
TRAIN_START_DATE: 2022-02-02 04:40:00
VAL_END_DATE: 2022-04-29 23:55:00

Starting CPCV optimization with:
drl algorithm: ppo
name_test: model
gpu_id: 0

Launched hyperparameter optimization with CPCV

TIMEFRAME 5m
TRAIN SAMPLES 20000
TRIALS NO. 50
N 5
K test groups 2
SPLITS 10

TRAIN SAMPLES 20000
VAL_SAMPLES 5000
TRAIN_START_DATE 2022-02-02 04:40:00
TRAIN_END_DATE 2022-04-12 15:15:00
VAL_START_DATE 2022-04-12 15:20:00
VAL_END_DATE 2022-04-29 23:55:00

TICKER LIST ['BTCUSDT', 'ETHUSDT']

/home/sb/anaconda3/envs/xucc/lib/python3.8/site-packages/optuna/samplers/_tpe/sampler.py:263: ExperimentalWarning: multivariate option is an experimental feature. The interface can change in the future.
warnings.warn(
[I 2023-02-21 11:43:29,394] A new study created in memory with name: no-name-79cf0045-f7b8-4b70-a4ef-86c712939aeb

LOADING DATA FOLDER: ./data/5m_25000

No. Train Samples: 14994

| Arguments Remove cwd: ./train_results/cwd_tests/model_CPCV_ppo_5m_50H_25k
################################################################################
ID Step maxR | avgR stdR avgS stdS | expR objC etc.
[W 2023-02-21 11:43:29,460] Trial 0 failed because of the following error: ValueError('operands could not be broadcast together with shapes (2,) (10,) ')
Traceback (most recent call last):
File "/home/sb/anaconda3/envs/xucc/lib/python3.8/site-packages/optuna/study/_optimize.py", line 213, in _run_trial
value_or_values = func(trial)
File "1_optimize_cpcv.py", line 332, in obj_with_argument
return objective(trial, name_test, model_name, cwd, res_timestamp, gpu_id)
File "1_optimize_cpcv.py", line 269, in objective
sharpe_bot, sharpe_eqw, drl_rets_tmp = train_and_test(trial, price_array, tech_array, train_indices,
File "/home/sb/Lab_4/FinRL_Crypto/function_train_test.py", line 33, in train_and_test
train_agent(price_array,
File "/home/sb/Lab_4/FinRL_Crypto/function_train_test.py", line 72, in train_agent
agent.train_model(model=model,
File "/home/sb/Lab_4/FinRL_Crypto/drl_agents/elegantrl_models.py", line 86, in train_model
train_and_evaluate(model)
File "/home/sb/Lab_4/FinRL_Crypto/train/run.py", line 40, in train_and_evaluate
trajectory = agent.explore_env(env, target_step)
File "/home/sb/Lab_4/FinRL_Crypto/drl_agents/agents/AgentPPO.py", line 74, in explore_one_env
next_s, reward, done, _ = env.step(get_a_to_e(ten_a)[0].numpy())
File "/home/sb/Lab_4/FinRL_Crypto/environment_Alpaca.py", line 113, in step
for index in np.where(actions < -self.minimum_qty_alpaca)[0]:
ValueError: operands could not be broadcast together with shapes (2,) (10,)
Traceback (most recent call last):
File "1_optimize_cpcv.py", line 364, in
optimize(name_test, name_model, gpu_id)
File "1_optimize_cpcv.py", line 345, in optimize
study.optimize(
File "/home/sb/anaconda3/envs/xucc/lib/python3.8/site-packages/optuna/study/study.py", line 400, in optimize
_optimize(
File "/home/sb/anaconda3/envs/xucc/lib/python3.8/site-packages/optuna/study/_optimize.py", line 66, in _optimize
_optimize_sequential(
File "/home/sb/anaconda3/envs/xucc/lib/python3.8/site-packages/optuna/study/_optimize.py", line 177, in _optimize_sequential
callback(study, frozen_trial)
File "1_optimize_cpcv.py", line 98, in save_best_agent
if study.best_trial.number != trial.number:
File "/home/sb/anaconda3/envs/xucc/lib/python3.8/site-packages/optuna/study/study.py", line 97, in best_trial
return copy.deepcopy(self._storage.get_best_trial(self._study_id))
File "/home/sb/anaconda3/envs/xucc/lib/python3.8/site-packages/optuna/storages/_in_memory.py", line 311, in get_best_trial
raise ValueError("No trials are completed yet.")
ValueError: No trials are completed yet.

When I'm running 1_optimize_cpcv.py, 1_optimize_kcv.py or 1_optimize_wf.py the optimising process gets always stuck after the first Trial (Trial 1) finished.

/Users/USER/Notebooks/FinRL_Crypto_CPCV_PBO-main/1_optimize_wf.py

10
TRAIN_START_DATE: 2022-02-02 04:40:00
VAL_END_DATE: 2022-04-29 23:55:00

Starting WF optimization with:
drl algorithm: ppo
name_test: model
gpu_id: 0

Launched hyperparameter optimization with K-Cross Validation

TIMEFRAME 5m
TRAIN SAMPLES 20000
TRIALS NO. 10
N 5
K slash 2
SPLITS 10

TRAIN SAMPLES 20000
VAL_SAMPLES 5000
TRAIN_START_DATE 2022-02-02 04:40:00
TRAIN_END_DATE 2022-04-12 15:15:00
VAL_START_DATE 2022-04-12 15:20:00
VAL_END_DATE 2022-04-29 23:55:00

TICKER LIST ['AAVEUSDT', 'AVAXUSDT', 'BTCUSDT', 'NEARUSDT', 'LINKUSDT', 'ETHUSDT', 'LTCUSDT', 'MATICUSDT', 'UNIUSDT', 'SOLUSDT']

/Users/USER/Notebooks/FinRL_Crypto_CPCV_PBO-main/.venv/lib/python3.9/site-packages/optuna/samplers/_tpe/sampler.py:263: ExperimentalWarning: multivariate option is an experimental feature. The interface can change in the future.
warnings.warn(
[I 2023-02-25 13:29:46,027] A new study created in memory with name: no-name-d7b91c2c-25dd-4592-9647-df312bfc1c45

LOADING DATA FOLDER: ./data/5m_25000

No. Train Samples: 19999

| Arguments Remove cwd: ./train_results/cwd_tests/model_WF_ppo_5m_10H_25k
################################################################################
ID Step maxR | avgR stdR avgS stdS | expR objC etc.
0 4.00e+04 0.92 |
0 4.00e+04 0.92 | 0.92 0.0 19997 0 | -0.00 3.55 -0.25 -0.46
0 8.00e+04 0.94 |
0 8.00e+04 0.94 | 0.94 0.0 19997 0 | -0.00 4.23 -0.30 -0.47
| UsedTime: 434 | SavedDir: ./train_results/cwd_tests/model_WF_ppo_5m_10H_25k

No. Test Samples: 4999

Test Finished!
episode_return: -0.10315243935446239

/Users/USER/Notebooks/FinRL_Crypto_CPCV_PBO-main/function_finance_metrics.py:275: RuntimeWarning: invalid value encountered in log
return np.log(1 + pct_returns + 1e-8)
[I 2023-02-25 13:37:01,484] Trial 0 finished with value: 25.167498967428035 and parameters: {'learning_rate': 0.03, 'batch_size': 512, 'gamma': 0.99, 'net_dimension': 1024, 'target_step': 37500, 'eval_time_gap': 60, 'break_step': 45000.0, 'lookback': 1, 'norm_cash': 0.000244140625, 'norm_stocks': 0.00390625, 'norm_tech': 3.0517578125e-05, 'norm_reward': 0.0009765625, 'norm_action': 10000}. Best is trial 0 with value: 25.167498967428035.

Found new best agent!

LOADING DATA FOLDER: ./data/5m_25000

No. Train Samples: 19999

| Arguments Remove cwd: ./train_results/cwd_tests/model_WF_ppo_5m_10H_25k
################################################################################
ID Step maxR | avgR stdR avgS stdS | expR objC etc.
0 4.00e+04 0.96 |
0 4.00e+04 0.96 | 0.96 0.0 19997 0 | -0.00 7.21 -0.28 -0.50
0 8.00e+04 0.96 | 0.95 0.0 19997 0 | -0.00 7.62 -0.30 -0.42
| UsedTime: 158 | SavedDir: ./train_results/cwd_tests/model_WF_ppo_5m_10H_25k

No. Test Samples: 4999

Test Finished!
episode_return: -0.21765804701412517

/Users/USER/Notebooks/FinRL_Crypto_CPCV_PBO-main/function_finance_metrics.py:275: RuntimeWarning: invalid value encountered in log
return np.log(1 + pct_returns + 1e-8)
[I 2023-02-25 13:39:40,762] Trial 1 finished with value: 27.337235005057845 and parameters: {'learning_rate': 0.015, 'batch_size': 3080, 'gamma': 0.99, 'net_dimension': 512, 'target_step': 37500, 'eval_time_gap': 60, 'break_step': 60000.0, 'lookback': 1, 'norm_cash': 0.000244140625, 'norm_stocks': 0.00390625, 'norm_tech': 3.0517578125e-05, 'norm_reward': 0.0009765625, 'norm_action': 10000}. Best is trial 1 with value: 27.337235005057845.

Found new best agent!

LOADING DATA FOLDER: ./data/5m_25000

No. Train Samples: 19999

| Arguments Remove cwd: ./train_results/cwd_tests/model_WF_ppo_5m_10H_25k
################################################################################
ID

I found that the codes in environment_alpaca.py has a paragraph like this

# FORCE 5% SELL every half day (30 min timeframe -> (24 * 2 / 2) * 30) for index in np.where(self.stocks_cooldown >= 48)[0]: sell_num_shares = self.stocks[index] * 0.05 self.stocks_cooldown[index] = 0 self.stocks[index] -= sell_num_shares self.cash += price[index] * sell_num_shares * (1 - self.sell_cost_pct)

why should the environment sell 5% for every stocks every half day? I found that there is no policy in any exchange that agent should sell 5% stocks.

Traceback (most recent call last):
File "0_dl_trainval_data.py", line 35, in
from processor_Binance import BinanceProcessor
File "/home/sb/Lab_4/FinRL_Crypto/processor_Binance.py", line 48, in
from config_api import *
ModuleNotFoundError: No module named 'config_api'