brendanator / atari-rl Goto Github PK

Thank you so much or sharing the project. That's right the paper I am reading these days.
While I was running main.py, it reported these errors in the main.py.
`raceback (most recent call last):
File "/Users/jiawen/anaconda3/envs/atari-rl/lib/python3.5/site-packages/absl/flags/_flag.py", line 181, in _parse
return self.parser.parse(argument)
File "/Users/jiawen/anaconda3/envs/atari-rl/lib/python3.5/site-packages/absl/flags/_argument_parser.py", line 114, in parse
type(argument)))
TypeError: flag value must be a string, found "<class 'tuple'>"

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "main.py", line 19, in
flags.DEFINE_string('input_shape', (84, 84), 'Rescale input to this shape')
File "/Users/jiawen/anaconda3/envs/atari-rl/lib/python3.5/site-packages/tensorflow/python/platform/flags.py", line 58, in wrapper
return original_function(*args, **kwargs)
File "/Users/jiawen/anaconda3/envs/atari-rl/lib/python3.5/site-packages/absl/flags/_defines.py", line 241, in DEFINE_string
DEFINE(parser, name, default, help, flag_values, serializer, **args)
File "/Users/jiawen/anaconda3/envs/atari-rl/lib/python3.5/site-packages/absl/flags/_defines.py", line 81, in DEFINE
DEFINE_flag(_flag.Flag(parser, serializer, name, default, help, **args),
File "/Users/jiawen/anaconda3/envs/atari-rl/lib/python3.5/site-packages/absl/flags/_flag.py", line 110, in init
self._set_default(default)
File "/Users/jiawen/anaconda3/envs/atari-rl/lib/python3.5/site-packages/absl/flags/_flag.py", line 211, in _set_default
self.default = self._parse(value)
File "/Users/jiawen/anaconda3/envs/atari-rl/lib/python3.5/site-packages/absl/flags/_flag.py", line 184, in _parse
'flag --%s=%s: %s' % (self.name, argument, e))
absl.flags._exceptions.IllegalFlagValueError: flag --input_shape=(84, 84): flag value must be a string, found "<class 'tuple'>"This also happened in the following code:flags.DEFINE_string('frameskip', 4, 'Number of frames to repeat actions for. '
'Can be int or tuple with min and max+1')`
Can I just change the type from string to integer? Thank you for your time!

Thanks for sharing such a cool thing!

But I have an another issue in master branch:

AttributeError: 'FastAtariEnv' object has no attribute '_buffer'

2018-05-21 21:24:06 MainThread: Populating replay memory
Traceback (most recent call last):
  File "main.py", line 171, in <module>
    tf.app.run()
  File "/home/alexshim/anaconda3/envs/py35_rl_gpu/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 44, in run
    _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  File "main.py", line 134, in main
    trainer.train()
  File "/home/alexshim/py_packs/atari-rl/agents/training.py", line 32, in train
    self.train_agent(session, self.agents[0])
  File "/home/alexshim/py_packs/atari-rl/agents/training.py", line 56, in train_agent
    agent.populate_replay_memory()
  File "/home/alexshim/py_packs/atari-rl/agents/agent.py", line 77, in populate_replay_memory
    if done: self.new_game()
  File "/home/alexshim/py_packs/atari-rl/agents/agent.py", line 20, in new_game
    observation, reward, done = self.atari.reset()
  File "/home/alexshim/py_packs/atari-rl/atari/atari.py", line 43, in reset
    self.last_frame = self.env.reset()
  File "/home/alexshim/py_packs/gym/gym/envs/atari/atari_env.py", line 100, in reset
    return self._get_obs()
  File "/home/alexshim/py_packs/gym/gym/envs/atari/atari_env.py", line 94, in _get_obs
    img = self._get_image()
  File "/home/alexshim/py_packs/atari-rl/atari/atari.py", line 110, in _get_image
    self.ale.getScreenRGB(self._buffer)  # says rgb but actually bgr
AttributeError: 'FastAtariEnv' object has no attribute '_buffer'

Can be it caused newer version of openai/atari? What version should I use?

Hi! It is really unclear from the article about pseudo-counts, but in appendix the authors clarify that they use location dependent CTS, which probably corresponds to LocationDependentDensityModel from https://github.com/brendanator/SkipCTS/blob/48af5c74ed43f724c61cdcf2e1a022f48c460ed7/python/tutorials/density_modelling_tutorial.ipynb

Hello!
Can you help me resolve an error?

(py35_rl_gpu) alexshim@AS-PC:~/py_packs/atari-rl$ python main.py --game BreakoutI tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcublas.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:126] Couldn't open CUDA library libcudnn.so.5. LD_LIBRARY_PATH:
I tensorflow/stream_executor/cuda/cuda_dnn.cc:3517] Unable to load cuDNN DSO
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcufft.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcurand.so.8.0 locally
/home/alexshim/py_packs/gym/gym/init.py:22: UserWarning: DEPRECATION WARNING: to improve load times, gym no longer automatically loads gym.spaces. Please run "import gym.spaces" to load gym.spaces on your own. This warning will turn into an error in a future version of gym.
warnings.warn('DEPRECATION WARNING: to improve load times, gym no longer automatically loads gym.spaces. Please run "import gym.spaces" to load gym.spaces on your own. This warning will turn into an error in a future version of gym.')
2018-05-21 17:05:30 MainThread: Creating network and training operations
Traceback (most recent call last):
File "main.py", line 145, in
tf.app.run()
File "/home/alexshim/anaconda3/envs/py35_rl_gpu/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 44, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "main.py", line 111, in main
trainer = Trainer(create_config())
File "/home/alexshim/py_packs/atari-rl/agents/training.py", line 16, in init
self.global_step, self.agents = factory.create_agents()
File "/home/alexshim/py_packs/atari-rl/networks/factory.py", line 57, in create_agents
global_step, train_ops = self.create_train_ops()
File "/home/alexshim/py_packs/atari-rl/networks/factory.py", line 104, in create_train_ops
for grad, var in grads:
UnboundLocalError: local variable 'grads' referenced before assignment

I am trying to train gpu version.
I will be very grateful.

Hi,

wow this is really a great repository 👍

Which method do you recommend to try first?

The optimality tightening paper makes some really impressive claims, about the speed that they learn. I was just wondering if you've managed to reproduce any of their results? I just saw a theano implementation by one of the authors here

Thanks for your advice,

Aj