rock-learning / bolero Goto Github PK
View Code? Open in Web Editor NEWBehavior Optimization and Learning for Robots
Home Page: https://rock-learning.github.io/bolero
License: Other
Behavior Optimization and Learning for Robots
Home Page: https://rock-learning.github.io/bolero
License: Other
The methods getstate() and setstate() are missing in the reps optimizer class.
When trying to pickle and controller object initialized with a reps optimizer, it throws an TypeError due to the logger object.
Adding (as in the other optimizers):
def __getstate__(self):
d = dict(self.__dict__)
del d["logger"]
return d
def __setstate__(self, d):
self.__dict__.update(d)
self.logger = get_logger(self, self.log_to_file, self.log_to_stdout)
should solve the problem.
With a preinstalled ROS it is not possible to successfully bootstrap BOLeRo. Building urdfdom fails, probably due to some collisions with ROS headers.
Further investigation of possible options to explicitly set the used dependencies/headers/libraries appear to be necessary.
Same applies to ApproxIK which also depends on urdfdom and which also fails to build when there is already a ROS installation on the system.
This happens even if ROS isn't activated (setup.bash not sourced).
While MARSEnvironmentHelper::init() tries to hand calc_ms to mars, if(marsPlugin->control->cfg) results in segementation fault. (master)
it works with 1.4.9
This is usually not a problem because the regularization coefficient is small. It is not straightforward to fix this because the upper-level policy that does the regression does not know which context feature is always 1 and hence represents the bias.
"""
...
learn_goal_velocities : bool, optional (default: True)
...
"""
def __init__(self, ... learn_goal_velocities=False, ...):
The docstring says it can be a string but it does not say which values are valid. One of them is 'quadratic'.
See #9
Loading an previously saved DMP model and executing the represented trajectory fails for some trajectories.
See test case with example trajectory in commit e18402b8e4d9b9d71aae7a7bf8e77c6d41b639b5 in forked repository.
While an synthetic sinus trajectory is correctly loaded, loading and executing non-synthetic trajectories produces bad results.
There must be an issue at C++ side as all the variables on python side are correctly loaded and identical.
The acceleration ydd quickly becomes quite large at the beginning of the trajectory execution.
https://papers.nips.cc/paper/5672-model-based-relative-entropy-stochastic-search.pdf, page 2
However, the episodic REPS algorithm uses a sample based
approximation of the KL-bound, which needs a lot of samples in order to be accurate. Moreover, a
typical problem of REPS is that the entropy of the search distribution decreases too quickly, resulting
in premature convergence.
The default initial mean is 0 and the default variance is 1. If a user defines the boundaries to be [0.1, 1], the search initial distribution will not be within the boundaries. We should inform the user in this case.
You should upload a pregenerated documentation webpage on github (gh-pages) and put a link into the README file.
DMP.imitate throws error, if dt has many decimal places, e.g. 1.0/60 (60 Hertz).
Policy Search
Policy Representation
Deep RL
Interesting Papers
at the moment, all dependencies are installed, all source code dependencies and the current master of bolero are built, the branch is built again after that and finally the tests will be run
possible improvements:
We have to set the meta class of Optimizer to ABCMeta like this is done for the ContextualOptimizer.
There is an old branch mars_wo_qt
. Is it still needed?
In the parameter description of the class DMPSequence, it is specified:
initial_weights : list
List of initial weight vectors for the DMPs
However, if initial_weights is set as a normal list, w.reshape would be called on an integer in line 109, which throws an AttributeError:
self.weights = [w.reshape(self.n_weights_per_dmp[i],
self.n_task_dims)
for w in self.initial_weights]
bolero/bolero/representation/dmp_sequence.py
Line 109 in 229f447
When building bolero the the file _ranking_svm.c is missing which has been removed in commit 99409c5
(venv) ~/git/bolero-dev/learning/bolero/build (master)$ make install -j
Scanning dependencies of target python_setup
non-existing path in 'bolero/utils': '_ranking_svm.c'
Warning: passing language='c++' to cythonize() is deprecated. Instead, put "# distutils: language=c++" in your .pyx or .pxd file(s)
running build_ext
running build_src
build_src
building extension "bolero.utils._ranking_svm" sources
building extension "bolero.wrapper._wrapper" sources
build_src: building npy-pkg config files
customize UnixCCompiler
customize UnixCCompiler using build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
building 'bolero.utils._ranking_svm' extension
compiling C sources
C compiler: /usr/bin/gcc -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC
compile options: '-I/media/hwiedemann/100F3CC0064B81ED/Home/git/bolero-dev/venv/lib/python3.5/site-packages/numpy/core/include -I/media/hwiedemann/100F3CC0064B81ED/Home/git/bolero-dev/venv/lib/python3.5/site-packages/numpy/core/include -I/usr/include/python3.5m -I/media/hwiedemann/100F3CC0064B81ED/Home/git/bolero-dev/venv/include/python3.5m -c'
extra options: '-O3'
gcc: _ranking_svm.c
gcc: error: _ranking_svm.c: No such file or directory
gcc: fatal error: no input files
compilation terminated.
error: Command "/usr/bin/gcc -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/media/hwiedemann/100F3CC0064B81ED/Home/git/bolero-dev/venv/lib/python3.5/site-packages/numpy/core/include -I/media/hwiedemann/100F3CC0064B81ED/Home/git/bolero-dev/venv/lib/python3.5/site-packages/numpy/core/include -I/usr/include/python3.5m -I/media/hwiedemann/100F3CC0064B81ED/Home/git/bolero-dev/venv/include/python3.5m -c _ranking_svm.c -o build/temp.linux-x86_64-3.5/_ranking_svm.o -MMD -MF build/temp.linux-x86_64-3.5/_ranking_svm.o.d -O3" failed with exit status 1
CMakeFiles/python_setup.dir/build.make:57: recipe for target 'python_setup' failed
make[2]: *** [python_setup] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/python_setup.dir/all' failed
make[1]: *** [CMakeFiles/python_setup.dir/all] Error 2
Makefile:138: recipe for target 'all' failed
make: *** [all] Error 2
I am trying to install bolero using bootstrap_bolero.sh inside a docker container.
The base image is python:3.7-stretch
.
The commands I run to install bolero are:
apt-get update
apt-get install -y sudo
pip install pyyaml cython numpy nose
wget https://raw.githubusercontent.com/rock-learning/bolero/master/bootstrap_bolero.sh
chmod +x bootstrap_bolero.sh
# Optionally tried to remove sudo, as the docker user is root anyways
# sed -i 's/sudo //g' bootstrap_bolero.sh
./bootstrap_bolero.sh
This seems to work fine (return code is 0), but in fact the output shows errors and bolero can not be imported after sourcing env.sh.
Errors:
- configure: learning/bolero/include
- configure: simulation/lib_manager
- configure: base/cmake
- configure: tools/configmaps
- configure: learning/bolero/src/optimizer/cmaes_optimizer
- configure: learning/bolero/src/bl_loader
- configure: learning/bolero/src/environment/function_approximation
- configure: learning/bolero/src/environment/mountain_car
- configure: learning/bolero/src/environment/cec13_test_functions
- configure: learning/bolero/src/optimizer/pso_optimizer
- configure: learning/bolero/src/representation/promp
- configure: learning/bolero/bolero/wrapper
- configure: learning/bolero/src/representation/dmp
- configure: learning/bolero/src/controller
- configure: learning/bolero
Installed packages:
[]
When trying to install by downloading the latest release from GitHub (https://github.com/rock-learning/bolero/releases/download/v1.0.0/bolero_release.zip), the install.sh runs some scripts which apparently are only Python 2 compatible, as they have print statements rather than using the print function, so that does not work well either.
Do you have any suggestions?
The full dockerfile is:
FROM python:3.7-stretch
RUN apt-get update \
&& apt-get install -y sudo \
&& pip install pyyaml cython numpy nose \
&& wget https://raw.githubusercontent.com/rock-learning/bolero/master/bootstrap_bolero.sh \
&& chmod +x bootstrap_bolero.sh \
# && sed -i 's/sudo //g' bootstrap_bolero.sh \
&& ./bootstrap_bolero.sh
COPY entrypoint.sh /entrypoint.sh
WORKDIR ./bolero-dev
ENTRYPOINT ["/entrypoint.sh"]
CMD nosetests bolero -sv
with entrypoint.sh being:
#!/bin/bash
set -e
source ./env.sh
exec "$@"
Main issues:
On Ubuntu you have to set these environment variables:
export CYTHON=cython3
export PYTHON=python3
@HWiese1980 told me that mars_environment depends on
Both are not listed in the manifest.xml. Should we add them? @malter
We've got a Heisenbug in ProMPs. :)
See for example this build: https://circleci.com/gh/rock-learning/bolero/88, from build stage:
learning/bolero/src/representation/promp build error
...
Errors:
- build: learning/bolero/src/representation/promp
However, unit tests of ProMPBehavior
seem to be OK:
test_promp_behavior.test_smoke ... ok
test_promp_behavior.test_dmp_default_promp ... ok
test_promp_behavior.test_promp_get_set_params ... ok
test_promp_behavior.test_promp_constructor_args ... ok
test_promp_behavior.test_promp_imitate ... ok
I tried to reproduce it:
af01/bolero_dependencies:latest
, cloned the repository, ran the build code; no errorThe error only seems to occur consistently during the first bootstrap in CircleCI. Even checked if that is an issue of pybob, but pybob only seems to report errors if the return code of make is != 0
which is not the case when I run it. Maybe @malter has another idea though.
When minimizing the dual function in REPS / C-REPS, why is the gradient approximated numerically rather than using the analytical expressions provided here? Is this for numerical stability?
The next release will be compatible with Windows and MacOS
test_environment.test_environments_follow_standard_protocol ... WARN: gym.spaces.Box autodetected dtype as <type 'numpy.float32'>. Please provide explicit dtype.
ok
test_objective_functions.test_optimum ... ok
test_objective_functions.test_input_validation ... ok
test_openaigym.test_discrete_input ... WARN: gym.spaces.Box autodetected dtype as <type 'numpy.float32'>. Please provide explicit dtype.
ok
test_openaigym.test_box_input ... WARN: gym.spaces.Box autodetected dtype as <type 'numpy.float32'>. Please provide explicit dtype.
WARN: gym.spaces.Box autodetected dtype as <type 'numpy.float32'>. Please provide explicit dtype.
dmp.imitate throws error if alpha==0.0 and n_steps < n_params
This is certainly related to
#define private public
in the test:
[ 64%] Building CXX object test/CMakeFiles/dmp_cpp_rigid_body_test.dir/test_rigid_body_dmp.cpp.o
In file included from /usr/include/c++/5/complex:45:0,
from /usr/local/include/eigen3/Eigen/Core:28,
from /usr/local/include/eigen3/Eigen/Geometry:4,
from /home/afabisch/Projekte/rock-bolero/learning/bolero/src/representation/dmp/src/QuaternionDmp.h:2,
from /home/afabisch/Projekte/rock-bolero/learning/bolero/src/representation/dmp/src/RigidBodyDmp.h:2,
from /home/afabisch/Projekte/rock-bolero/learning/bolero/src/representation/dmp/test/test_rigid_body_dmp.cpp:8:
/usr/include/c++/5/sstream:300:7: error: ‘struct std::__cxx11::basic_stringbuf<_CharT, _Traits, _Alloc>::__xfer_bufptrs’ redeclared with different access
struct __xfer_bufptrs
when I run ./bootstrap_bolero.sh to install bolero, it went into error while building dmp.
Sine it did not print the reason why this failed, I can't do anything to fix this.
Below is what it looks like:
Install: ['learning/bolero/bolero/wrapper', 'learning/bolero/src/controller', 'learning/bolero/src/representation/dmp']
learning/bolero/src/controller configured
learning/bolero/bolero/wrapper configured
learning/bolero/bolero/wrapper installed
learning/bolero/src/representation/dmp configured
learning/bolero/src/controller installed
learning/bolero/src/representation/dmp build error
Install: ['learning/bolero']
learning/bolero configured
learning/bolero build error
Errors:
these dependencies are not handled correctly in the installation process at the moment:
python-sklearn is dep from: learning/bolero
python-matplotlib is dep from: learning/bolero
python-scipy is dep from: learning/bolero
Under python3 the import from ._wrapper import CppBLLoader
in bolero/wrapper/__init__.py fails.
Therefore I can't create a controller instance from a dict. The config works for bolero_controller
from bolero.controller import Controller
cfg = yaml.load(open("../learning_config.yml"))
Controller(cfg)
Did I miss to install something?
exported PYTHON and CYTHON to version 3, did cmake .., make install in build, also did python3 setup.py install
This must be a scaling problem. Variance 1 seems to work, everything else breaks.
learning_config.yml
missing:
I added some unit tests but there are still some things missing, for example, reading and saving configuration files.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.