Code Monkey home page Code Monkey logo

meee's Introduction

Instruction to reproduce MEEE

Code to reproduce the experiments in Sample Efficient Reinforcement Learning via Model-Ensemble Exploration and Exploitation[abs].

It is noteworthy that our code is mainly based on MBPO, and we refer interested readers to the original code base MBPO for more details.

Installation

  1. Install MuJoCo 2.0 at ~/.mujoco/mujoco200 and copy your license key to ~/.mujoco/mjkey.txt, for example, you need to install the following dependencies first for Linux platform:
sudo yum install patchelf
sudo yum install mesa-libGL-devel mesa-libGLU-devel
sudo yum install mesa-libOSMesa-devel
sudo yum install mesa-libOSMesa
sudo yum install glfw
sudo yum install mesa-libGL
sudo yum install openmpi-devel
  1. Create a conda environment and install dependencies in requirements.txt
cd code_meee
conda create -n "your_env_name" python=3.6
conda activate "your_env_name"
# install cuda to suport tf-gpu==1.13.1
conda install cudatoolkit==10.0.130 
conda install cudnn==7.6.5
pip install -r requirements.txt

Usage

Configuration files can be found in examples/config. Use the following command to conduct experiment on Humanoid-v2:

python main.py run_local examples.development --config=examples.config.humanoid.1 --trial-gpus=1

Currently only running locally is supported, so just keep the run_local and examples.development arguments. examples.config.humanoid.1 determines the configuration file you want to use, and --trial-gpus=1 indicate that you would like to experiment with one Nvidia GPU, you could change the experiment environment and GPU used by modifying relative arguments.

Logging

The results can be found in the default directory log_dir=~/ray_meee/, you could also specify the directory in examples/config/configuration_files.

Citation

If you use this code or results in your paper, please cite our work as:

@inproceedings{yao2021sample,
  title={Sample efficient reinforcement learning via model-ensemble exploration and exploitation},
  author={Yao, Yao and Xiao, Li and An, Zhicheng and Zhang, Wanpeng and Luo, Dijun},
  booktitle={2021 IEEE International Conference on Robotics and Automation (ICRA)},
  pages={4202--4208},
  year={2021},
  organization={IEEE}
}

License

The code in this repository is released under the MIT license as found in the LICENSE file.

meee's People

Contributors

dependabot[bot] avatar yaoyao1995 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

meee's Issues

No module named 'softlearning.misc'

Hello! I've tried reproducing your experiments, but got the error mentioned in the issue title.

Could you please commit the missing module?

KeyError: 'WeightedReplayPool'

Hi,

I am reproducing your work and tried look around in mbpo softlearning and also MEEE's repo but I cannot see WeightedReplayPool in POOL_CLASSES anywhere. Could you please help with this? Thank you so much!

Traceback (most recent call last):
  File "/cmlscratch/wwongkam/env/meee/lib/python3.6/site-packages/ray/tune/trial_runner.py", line 389, in _process_events
    result = self.trial_executor.fetch_result(trial)
  File "/cmlscratch/wwongkam/env/meee/lib/python3.6/site-packages/ray/tune/ray_trial_executor.py", line 252, in fetch_result
    result = ray.get(trial_future[0])
  File "/cmlscratch/wwongkam/env/meee/lib/python3.6/site-packages/ray/worker.py", line 2288, in get
    raise value
ray.exceptions.RayTaskError: �[36mray_ExperimentRunner:train()�[39m (pid=1492472, host=cml12.umiacs.umd.edu)
  File "/cmlscratch/wwongkam/env/meee/lib/python3.6/site-packages/ray/tune/trainable.py", line 150, in train
    result = self._train()
  File "/cmlscratch/wwongkam/MEEE/examples/development/main.py", line 86, in _train
    self._build()
  File "/cmlscratch/wwongkam/MEEE/examples/development/main.py", line 54, in _build
    get_replay_pool_from_variant(variant, training_environment))
  File "/cmlscratch/wwongkam/MEEE/softlearning/replay_pools/utils.py", line 26, in get_replay_pool_from_variant
    replay_pool = POOL_CLASSES[replay_pool_type](
KeyError: 'WeightedReplayPool'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.