Portions of the code in saclag.py and wcsac.py are adapted from Safety Starter Agents.
Warning: If you want to use the WCSAC algorithm in Safety Gym, make sure to install Safety Gym according to the instructions on the Safety Gym repo.
The various utilities here are copied over from Spinning Up in Deep RL.
Two custom environments (StaticEnv-v0 and DynamicEnv-v0) are created within wcsac.py and saclag.py.
Two ways to install this package:
To install this package:
git clone https://github.com/AlgTUDelft/WCSAC.git
cd WCSAC/
pip install -e .
So you can install Safety Starter Agents, and add wcsac.py and saclag.py of our package to /path/to/safety-starter-agents/safe_rl/sac.
Then you can follow the instructions on the Safety Starter Agents to use our algorithms as using their given baselines.
Example: If you want to test WCSAC with risk level 0.5 in StaticEnv-v0, run:
cd /path/to/wcsac
python wcsac.py --env 'StaticEnv-v0' -s SEED --cl 0.5 --cost_lim d --logger_kwargs_str '{"output_dir": "./xxx"}'
where SEED is the random seed, d is the real-world safety threshold, and '{"output_dir": "./xxx"}' indicates where to store the data.
If you use WCSAC code in your paper, please cite:
@inproceedings{WCSAC,
title = {{WCSAC: Worst-Case Soft Actor Critic for Safety-Constrained Reinforcement Learning}},
author = {
Yang, Qisong and
Sim{\~{a}}o, Thiago D. and
Tindemans, Simon H. and
Spaan, Matthijs T. J.
},
booktitle = {Thirty-Fifth AAAI Conference on Artificial Intelligence},
pages = {10639--10646},
year = {2021}
}
@article{WCSAC-IQN,
author = {Qisong Yang and
Thiago D. Sim{\~{a}}o and
Simon H. Tindemans and
Matthijs T. J. Spaan},
title = {Safety-constrained reinforcement learning with a distributional safety
critic},
journal = {Mach. Learn.},
volume = {112},
number = {3},
pages = {859--887},
year = {2023}
}
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent