ikostrikov / jaxrl Goto Github PK
View Code? Open in Web Editor NEWJAX (Flax) implementation of algorithms for Deep Reinforcement Learning with continuous action spaces.
License: MIT License
JAX (Flax) implementation of algorithms for Deep Reinforcement Learning with continuous action spaces.
License: MIT License
Hey @ikostrikov really cool project!
I was trying to install it but hit some installation difficulties. It should be an easy fix if I look around more, but maybe it's worth introducing a proper dependency manager like poetry
. I filed PR #20 to close this issue. Let me know if you'd be interested in it and I can add more docs :)
➜ jaxrl git:(main) ✗ . /home/costa/.cache/pypoetry/virtualenvs/jaxrl-m03KgMHV-py3.9/bin/activate
(jaxrl-m03KgMHV-py3.9) ➜ jaxrl git:(main) ✗ pip install -e .
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Obtaining file:///home/costa/Documents/go/src/github.com/vwxyzjn/jaxrl
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Installing collected packages: jaxrl
Running setup.py develop for jaxrl
ERROR: Command errored out with exit status 1:
command: /home/costa/.cache/pypoetry/virtualenvs/jaxrl-m03KgMHV-py3.9/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/costa/Documents/go/src/github.com/vwxyzjn/jaxrl/setup.py'"'"'; __file__='"'"'/home/costa/Documents/go/src/github.com/vwxyzjn/jaxrl/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
cwd: /home/costa/Documents/go/src/github.com/vwxyzjn/jaxrl/
Complete output (3 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'setuptools'
----------------------------------------
ERROR: Command errored out with exit status 1: /home/costa/.cache/pypoetry/virtualenvs/jaxrl-m03KgMHV-py3.9/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/costa/Documents/go/src/github.com/vwxyzjn/jaxrl/setup.py'"'"'; __file__='"'"'/home/costa/Documents/go/src/github.com/vwxyzjn/jaxrl/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.
In these lines, because the assert statement is broken into multiple lines and hence covered with parentheses, the assert evaluates the tuple itself. So it will always evaluate to True.
jaxrl/jaxrl/networks/common.py
Lines 80 to 81 in 3674862
jaxrl/jaxrl/networks/common.py
Lines 89 to 90 in 3674862
This can be fixed by changing it to
assert loss_fn is not None or grads is not None, \
'Either a loss function or grads must be specified.'
assert has_aux, \
'When grads are provided, expects no aux outputs.'
TF is a heavy library that might need a lot of dependencies and time to install. Removing the dependency on tensorflow_probability is not essential but might be good too.
Great work btw
Right now it indicates whether the env is done due to a timelimit, while in the update we use 0.99.
Hi,
Thanks a lot for the great implementation!
I have a question regarding the temperature, I saw that in
https://github.com/ikostrikov/jax-rl/blob/main/jax_rl/agents/sac/temperature.py#L16
You are computing the temperature (entropy coefficient) by this instead of directly taking an exp on the log-temperature. Is there a particular reason that you implement it this way as compared to other implementation (for example, the pytorch DrQ, soft learning etc.)
Thanks!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.