The average Episodic Return and Average Loss is nan about ppo-for-beginners HOT 4 CLOSED

# Collect episodic length and Rewards
batch_lens.append(ep_t + 1)
batch_rews.append(ep_rews)

#Reshape data as tensors
batch_obs = torch.tensor(batch_obs, dtype=torch.float)
batch_acts  = torch.tensor(batch_acts, dtype=torch.float)
batch_log_probs = torch.tensor(batch_log_probs, dtype=torch.float)

# ALG STEP 4

batch_rtgs = self.compute_rtgs(batch_rews)

print(f'Batch rewards before logger======={batch_rews}')

# Log the episodic returns and episodic lengths in this batch.
self.logger['batch_rews'] = batch_rews
self.logger['batch_lens'] = batch_lens

# Return the batch data
return batch_obs, batch_acts, batch_log_probs, batch_rtgs, batch_lens

#Convert the reward-to-go into a tensor
batch_rtgs = torch.tensor(batch_rtgs, dtype=torch.float)

return batch_rtgs

Model information ====== <ppo.PPO object at 0x7fa61287c358>
Learning... Running 200 timesteps per episode, 2048 timesteps per batch for a total of 1000000 timesteps

-------------------- Iteration #1 --------------------
Average Episodic Length: 200.0
Average Episodic Return: -1358.09
Average Loss: -0.00129
Timesteps So Far: 2200
------------------------------------------------------


-------------------- Iteration #2 --------------------
Average Episodic Length: 200.0
Average Episodic Return: -1172.53
Average Loss: -0.001
Timesteps So Far: 4400
------------------------------------------------------

^C