Code Monkey home page Code Monkey logo

elegantrl's Introduction

ElegantRL β€œε°ι›…β€: Massively Parallel Library for Cloud-native Deep Reinforcement Learning

Downloads Downloads Python 3.6 PyPI



ElegantRL is developed for practitioners with the following advantages:

  • Cloud-native: follows a cloud-native paradigm through microservice architecture and containerization, supporting ElegantRL-Podracer and FinRL-Podracer.

  • Scalable: fully exploits the parallelism of DRL algorithms at multiple levels, making it easily scale out to hundreds or thousands of computing nodes on a cloud platform, say, a DGX SuperPOD platform with thousands of GPUs.

  • Elastic: allows to elastically and automatically allocate computing resources on the cloud.

  • Lightweight: the core codes <1,000 lines (check Elegantrl_Helloworld).

  • Efficient: in many testing cases (single GPU/multi-GPU/GPU cloud), we find it more efficient than Ray RLlib.

  • Stable: much much much more stable than Stable Baselines 3 by utilizing various ensemble methods.

ElegantRL implements the following model-free deep reinforcement learning (DRL) algorithms:

  • DDPG, TD3, SAC, PPO, REDQ for continuous actions,
  • DQN, Double DQN, D3QN, SAC for discrete actions,
  • QMIX, VDN, MADDPG, MAPPO, MATD3 for multi-agent environment.

For the details of DRL algorithms, please check out the educational webpage OpenAI Spinning Up.

ElegantRL supports the following simulators:

  • Isaac Gym for massively parallel simulation,
  • OpenAI Gym, MuJoCo, PyBullet, FinRL for benchmarking.

β€œε°ι›…β€ζΊδΊŽγ€Šθ―—η»Β·ε°ι›…Β·ιΉ€ιΈ£γ€‹οΌŒζ—¨εœ¨γ€Œδ»–ε±±δΉ‹ηŸ³οΌŒε―δ»₯ζ”»ηŽ‰γ€γ€‚

Contents

News

ElegantRL-Helloworld

For beginners, we maintain ElegantRL-Helloworld as a tutorial. Its goal is to get hands-on experience with ELegantRL.

One sentence summary: an agent (agent.py) with Actor-Critic networks (net.py) is trained (run.py) by interacting with an environment (env.py).

File Structure

  • elegantrl # main folder
    • envs, # a collection of environments
    • agent.py, # DRL algorithms
    • config.py, # configurations (hyper-parameter)
    • demo.py, # a collection of demos
    • evaluator.py # the evaluator class
    • net.py # a collection of network architectures
    • replay_buffer.py # the buffer class
    • run.py # training loop
  • elegantrl_helloworld # tutorial version
    • env.py,
    • agent.py,
    • demo.py,
    • net.py
    • run.py
  • examples # a collection of example codes
  • ready-to-run Google-Colab notebooks
    • quickstart_Pendulum_v1.ipynb
    • tutorial_BipedalWalker_v3.ipynb
    • tutorial_Creating_ChasingVecEnv.ipynb
    • tutorial_LunarLanderContinuous_v2.ipynb

Experimental Demos

More efficient than Ray RLlib

Experiments on Ant (MuJoCo), Humainoid (MuJoCo), Ant (Isaac Gym), Humanoid (Isaac Gym) # from left to right

ElegantRL fully supports Isaac Gym that runs massively parallel simulation (e.g., 4096 sub-envs) on one GPU.

More stable than Stable-baseline 3

Experiment on Hopper-v2 # ElegantRL achieves higher average rewards and much smaller variance (average over 10 runs)

Requirements

Necessary:
| Python 3.6+     |           
| PyTorch 1.6+    |    

Not necessary:
| Numpy 1.18+     | For ReplayBuffer. Numpy will be installed along with PyTorch.
| gym 0.17.0      | For env. Gym provides tutorial env for DRL training. (env.render() bug in gym==0.18 pyglet==1.6. Change to gym==0.17.0, pyglet==1.5)
| pybullet 2.7+   | For env. We use PyBullet (free) as an alternative of MuJoCo (not free).
| box2d-py 2.3.8  | For gym. Use pip install Box2D (instead of box2d-py)
| matplotlib 3.2  | For plots. 

pip3 install gym==0.17.0 pybullet Box2D matplotlib

To install StarCraftII env,
bash ./elegantrl/envs/installsc2.sh
pip install -r sc2_requirements.txt

Citation:

To cite this repository:

@misc{erl,
  author = {Liu, Xiao-Yang and Li, Zechu and Wang, Zhaoran and Zheng, Jiahao},
  title = {{ElegantRL}: Massively Parallel Framework for Cloud-native Deep Reinforcement Learning},
  year = {2021},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/AI4Finance-Foundation/ElegantRL}},
}

elegantrl's People

Contributors

bruceyanghy avatar cryptocoinserver avatar csbobby avatar daijing5763 avatar emkan90 avatar eric-lan0 avatar everssun avatar geekpineapple avatar guang384 avatar hmomin avatar idiomaticrefactoring avatar ludel avatar orionzou avatar rayrui312 avatar shixun404 avatar supersglzc avatar xiao000l avatar yangletliu avatar yonv1943 avatar zhangaipi avatar zhumingpassional avatar ziyixia avatar zywang624 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.