maxsobolmark / hardrlwithyoutube Goto Github PK

I was able to train the embedding reward, but when I ran PPO it ended with a reward of zero. Were you able to get non-zero rewards on Montezuma's Revenge?

| approxkl | 0.00054461794 |
| clipfrac | 0.1274414 |
| eplenmean | 1.38e+03 |
| eprewmean | 0 |
| explained_variance | 0.888 |
| fps | 1082 |
| nupdates | 1950 |
| policy_entropy | 2.5611527 |
| policy_loss | -0.0030562507 |
| serial_timesteps | 249600 |
| time_elapsed | 9.28e+03 |
| total_timesteps | 9984000 |
| value_loss | 0.1622782 |

I followed the procedure to download videos. After that when I tried to train the featurizer, it happened this.

I tried to figure out why occurred to this error.

this means Video1 and Video3 can't successfully preprocess and only Video2 to train the featurizer.

And after I found that empty range error due to this code in TDC model, which as said in the paper.

after I changed this possible_frames_end to 80 from 200, It trains the featurizer successfully but by only Video2.

So what should I do to solve this? Thanks!

After training, when I try to visualize the embeddings on the tensorboard,
the tensorboard fails to load the data and is stuck at this:

I am running it on TensorBoard 1.12.0