0xangelo / raylab Goto Github PK
View Code? Open in Web Editor NEWReinforcement learning algorithms in RLlib
License: MIT License
Reinforcement learning algorithms in RLlib
License: MIT License
Currently we use a very ad-hoc procedure for scaling the quadratic component of NAF when used for exploration:
https://github.com/angelolovatto/raylab/blob/9820275b17ee085e1955a6d845c0bdf61333f8da/raylab/algorithms/naf/naf_policy.py#L150-L155
A possibly better alternative would be to scale it based on the desired average action stddev. Something like:
scale_tril * (1.0 / average_sttdev(scale_tril)) * desired_avg_stddev
That way we could contain the magnitude of the MultivariateNormal
precision matrix while still allowing some freedom for diversifying the action's covariances.
When running raylab experiment "PG" --config examples/PG/cartpole_defaults.py
I get the following error:
Traceback (most recent call last):
File "/home/palenicek/miniconda3/envs/raylab_test/bin/raylab", line 8, in <module>
sys.exit(raylab())
File "/home/palenicek/miniconda3/envs/raylab_test/lib/python3.7/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/home/palenicek/miniconda3/envs/raylab_test/lib/python3.7/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/home/palenicek/miniconda3/envs/raylab_test/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/palenicek/miniconda3/envs/raylab_test/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/palenicek/miniconda3/envs/raylab_test/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/home/palenicek/miniconda3/envs/raylab_test/lib/python3.7/site-packages/raylab/cli/utils.py", line 164, in wrapped
return func(*args, tune_kwargs=tune_kwargs, **kwargs)
File "/home/palenicek/miniconda3/envs/raylab_test/lib/python3.7/site-packages/raylab/cli/utils.py", line 19, in wrapped
return func(*args, **kwargs)
File "/home/palenicek/miniconda3/envs/raylab_test/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/palenicek/miniconda3/envs/raylab_test/lib/python3.7/site-packages/raylab/cli/utils.py", line 51, in wrapped
process_tune_kwargs(ctx, **tune_kwargs)
File "/home/palenicek/miniconda3/envs/raylab_test/lib/python3.7/site-packages/raylab/cli/utils.py", line 172, in process_tune_kwargs
delete_if_necessary(ctx, osp.join(local_dir, name))
File "/home/palenicek/miniconda3/envs/raylab_test/lib/python3.7/posixpath.py", line 94, in join
genericpath._check_arg_types('join', a, *p)
File "/home/palenicek/miniconda3/envs/raylab_test/lib/python3.7/genericpath.py", line 153, in _check_arg_types
(funcname, s.__class__.__name__)) from None
TypeError: join() argument must be str or bytes, not 'NoneType'
I tried installing ray lab both using pip, as well as from source master and the last stable release version 0.10.0. I always get this error.
I am not sure, what the problem is. From the documentation I am also not 100% sure on how raylab should be used. I tried running the cli commands, as well as the /example files directly. All of which gave me the above error.
Any ideas on how to fix this issue?
Hello,
We've tried to activate or update your repository on Depfu and couldn't find any supported dependency files. If we were to guess, we would say that this is not actually a project Depfu supports and has probably been activated by error.
Please note that Depfu currently only searches for your dependency files in the root folder. We do support monorepos and non-root files, but don't auto-detect them. If that's the case with this repo, please send us a quick email with the folder you want Depfu to work on and we'll set it up right away!
Please note that using the "All Repositories" setting doesn't make a lot of sense with Depfu.
Please let us know by sending an email to [email protected].
This is an automated issue by Depfu. You're getting it because someone configured Depfu to automatically update dependencies on this project.
The title is self-explanatory. The specific classes are the ones below. It would be nice for users to understand these problems without diving too deeply into the code.
@thiagopbueno, perhaps you have some description of these tasks lying around?
Our current implementation of Reservoir only samples one Gamma random variable per reservoir in the config. However, we should sample batches of rain
variables inputting batches of states int the transition_fn
. It does not crash currently because of broadcasting rules, as we add the rain samples to the batch of states. However, that shares random variables across batches of transitions, which is not ideal.
Furthermore, we're calculating the log-likelihood as if state and batch dimensions were uncorrelated, yielding a B x N matrix logp
, where B is the batch size and N is the state dimension. We should use PyTorch's Independent
distribution to squeeze logp
across the state dimension, which isn't hard to do. An example can be found in raylab.distributions.DiagMultivariateNormal
. I'll get right into it.
A last concern is about the nature of the log-likelihood itself. Even though rain
is the only random variable involved in the transition, it is not clear that the transition's logp
is the logp
of the rain
variable only. Although, since all subsequent modifications to the rain
variable are additive, I assume that the change of variables theorem would give us the same pdf as the original rain
variable anyway. @thiagopbueno, thoughts?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.