I have a problem, i cannot run my code. According to this project, I need to use Python version 3.9.8 but it still wont work even when i have all the packages in requirements.txt installed. I don't know how to solve this, can someone please help?

And when i run this line, i get an error:

pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html

This repo is pretty awesome. I'm trying to run a basic demo, but the training process seems not working at all (rewards don't converge at all). However, the agent still outperforms B&H a lot.(even when the reward is negative! ) I'm confused by this situation. Is there an explanation about this?

The graph is rewards with training epochs=50.

Does anyone have a solution? I tried to run the return statement only if the length of rate_of_return is not null, but then i get a followup error like IndexError.

I'm trying to run this code:
`#Read RL Agent experiment
data_test_rlagent = dataTest_patternBased.data if dataTest_patternBased is not None else None

rlAgent = RLTrain(dataTrain_patternBased.data, data_test_rlagent, data_loader.patterns, DATASET_NAME, n=n, num_iteration=num_iteration, gamma=gamma, alpha=alpha, epsilon=epsilon)

rlAgent.training()
rlAgent.write_to_file()

#rlAgent.read_from_file('GOOGL-TRAIN_TEST_SPLIT(True)-NUM_ITERATIONS10000-N_STEP10-GAMMA1-ALPHA0.1-EPSILON0.1-EXPERIMENT(1).pkl')
#rlAgent.read_from_file('AAPL-TRAIN_TEST_SPLIT(True)-NUM_ITERATIONS1000-N_STEP10-GAMMA1-ALPHA0.1-EPSILON0.1-EXPERIMENT.pkl')
#rlAgent.read_from_file('BTC-USD-TRAIN_TEST_SPLIT(True)-NUM_ITERATIONS10000-N_STEP10-GAMMA0.9-ALPHA0.1-EPSILON0.1-EXPERIMENT.pkl')
#rlAgent.read_from_file('KSS-TRAIN_TEST_SPLIT(True)-NUM_ITERATIONS10000-N_STEP10-GAMMA0.9-ALPHA0.1-EPSILON0.1-EXPERIMENT(1).pkl')

ev_rlAgent = rlAgent.test(test_type= 'train')
print('train')
ev_rlAgent.evaluate()
rlAgent_portfolio_train = ev_rlAgent.get_daily_portfolio_value()
ev_rlAgent = rlAgent.test(test_type= 'test')
print('test')
ev_rlAgent.evaluate()
rlAgent_portfolio_test = ev_rlAgent.get_daily_portfolio_value()

model_kind = f'RL'
#model_kind = f'RL-{n}'

add_train_portfo(model_kind, rlAgent_portfolio_train)
add_test_portfo(model_kind, rlAgent_portfolio_test)`

I'm trying to run the Main.py, it runs perfectly, the only issue here is that it writes new data(BTC-USD) to the file data_processed.csv but it does not plot it into a visual chart (jpg file). I've set load_from_file=False and it still doesn't work, what am I doing wrong?

`

BTC-USD

DATASET_NAME = 'BTC-USD'
DATASET_FOLDER = r'BTC-USD'
FILE = r'BTC-USD.csv'
data_loader = YahooFinanceDataLoader(DATASET_FOLDER, FILE, '2021-01-10', load_from_file=False)
transaction_cost = 0.0
`

I searched a lot but couldn't find a proper way to fix this.

I believe Batch Normalization cause the problem. Do you have any advice about it ?

def get_reward(self, action):
    """
    @param action: based on the action taken it returns the reward
    @return: reward
    """

    reward_index_first = self.current_state_index + self.start_index_reward
    reward_index_last = self.current_state_index + self.start_index_reward + self.n_step \
        if self.current_state_index + self.n_step < len(self.states) else len(self.close_price) - 1

    p1 = self.close_price[reward_index_first]
    p2 = self.close_price[reward_index_last]

    reward = 0
    if action == 0 or (action == 1 and self.own_share):  # Buy Share or Hold Share
        reward = ((1 - self.trading_cost_ratio) ** 2 * p2 / p1 - 1) * 100  # profit in percent
    elif action == 2 or (action == 1 and not self.own_share):  # Sell Share or No Share
        # consider the transaction in the reverse order
        reward = ((1 - self.trading_cost_ratio) ** 2 * p1 / p2 - 1) * 100

    return reward

Sorry, the function is true.

Warning Risk of data leak Do not use minmax_scale unless you know what you are doing. A common mistake is to apply it to the entire data before splitting into training and test sets. This will bias the model evaluation because information would have leaked from the test set to the training set. In general, we recommend using MinMaxScaler within a Pipeline in order to prevent most risks of data leaking: pipe = make_pipeline(MinMaxScaler(), LogisticRegression()).

from https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.minmax_scale.html .

Hi, when I run main.py passes through these two files .
the function os.makedirs( ) points out these filename or extension is too long to create.
"[WinError 206] The filename or extension is too long. "

• EncoderDecoderAgent/BaseTrain.py (line 89)
  self.PATH = os.path.join(Path(os.path.abspath(os.path.dirname(file))).parent,
  f'Results/{self.DATASET_NAME}/'
  f'{self.model_kind}; '
  f'DATA_KIND({self.data_train.data_kind}); '
  f'BEGIN_DATE({self.begin_date}); '
  f'END_DATE({self.end_date}); '
  f'SPLIT_POINT({self.split_point}); '
  f'WindowSize({self.window_size}); '
  f'BATCH_SIZE{self.BATCH_SIZE}; '
  f'GAMMA{self.GAMMA}; '
  f'REPLAY_MEMORY_SIZE{self.ReplayMemorySize}; '
  f'TARGET_UPDATE{self.TARGET_UPDATE}; '
  f'N_STEP{self.n_step}')

    if not os.path.exists(self.PATH):
        os.makedirs(self.PATH)
  

• DeepRLAgent/BaseTrain.py (line 100)
  self.PATH = os.path.join(Path(os.path.abspath(os.path.dirname(file))).parent,
  f'Results/{self.DATASET_NAME}/'
  f'{self.model_kind}; '
  f'DATA_KIND({self.data_train.data_kind}); '
  f'BEGIN_DATE({self.begin_date}); '
  f'END_DATE({self.end_date}); '
  f'SPLIT_POINT({self.split_point}); '
  f'StateMode({self.state_mode}); '
  f'WindowSize({self.window_size}); '
  f'BATCH_SIZE{self.BATCH_SIZE}; '
  f'GAMMA{self.GAMMA}; '
  f'REPLAY_MEMORY_SIZE{self.ReplayMemorySize}; '
  f'TARGET_UPDATE{self.TARGET_UPDATE}; '
  f'N_STEP{self.n_step}')

    if not os.path.exists(self.PATH):
        os.makedirs(self.PATH)
  

I would appreciate it if you can tell me where the problem is.
THANKS!