Comments (3)
Yeah usually its set to 10000 but had set to 20000 when trained the BeamriderNoFrameskip-v4 as that was limiting the score within 30mins of training and time limit in gym is actually 100,000 for that env. Forgot to change back.
This argument is to stop a stuck game from just running out all 100,000 steps to finish and instead start new game sooner as a lot of games only need 10000 step to complete.
if you see environment.py
if lives < self.lives and lives > 0:
# for Qbert sometimes we stay in lives == 0 condtion for a few frames
# so its important to keep lives > 0, so that we only reset once
# the environment advertises done.
done = True
self.was_real_done = False
this matches same functionality
sorry thought I had commented there it was a quick ugly hack and meant to clean that up. Had End of lives as episodic for env and model before but quickly change to just have end of lives episodic for model only to check performance matched fine either way
from rl_a3c_pytorch.
Yes, your concern is right.
If a user sets max-episode-length <= gym's internal max_episode_length, there is no problem. But if a user sets max-episode-length > gym's internal max_episode_length, it's always gym's internal max_episode_length to be reached first and gym terminates the episode. Now info['ale.lives'] > 0 and done = True and player.max_length = False.
Your code in test.py will execute:
if player.done and player.info['ale.lives'] > 0 and not player.max_length:
state = player.env.reset()
player.eps_len += 2
player.state = torch.from_numpy(state).float()
if gpu_id >= 0:
with torch.cuda.device(gpu_id):
player.state = player.state.cuda()
which is the branch that assumes episode hasn't terminated.
It would be good if you can remind users to set max-episode-length <= gym's internal max_episode_length.
from rl_a3c_pytorch.
haha yeah I guess you could set it over gym limit but it will still reset environment on max-episode-length setting which is fine for now. I guess I assumed it was obvious to set less or equal to gym setting. I'm fine with it for now. maybe will change but more likely will revert to previous wrapper settings as was temporarily trying to be more like baselines for comparison but not a fan of them
from rl_a3c_pytorch.
Related Issues (20)
- Why ensure_shared_grads HOT 1
- NotImplementedError HOT 1
- Pretrained models HOT 6
- Quick question on batch processing HOT 4
- eps for Adam HOT 2
- plot rewards as a function of number of timesteps HOT 1
- Stuck when training in MsPacman-v0 HOT 7
- Reward Smoothing HOT 2
- Need for trained models HOT 2
- Cannot import test HOT 1
- Question about Test function HOT 1
- Is there any necessary to lock when update params? HOT 1
- How can I let the training automatically stop after a given number of episodes or after a given period of time? HOT 1
- Clarification needed regarding num_workers HOT 2
- question about trained models HOT 1
- UserWarning: This overload of add_ is deprecated
- Need a model, thank you HOT 1
- The links to the Gym environment evaluations is 404.
- run a3c on 8 cpus, it still slow. HOT 1
- Hyperparameters for training
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rl_a3c_pytorch.