- Please pay attention to the version of SC2 you are using for your experiments. 
- Performance is *not* always comparable between versions. 
- The results in PYMARL-RLC use SC2.4.10.

PyMARL-RLC includes implementations of the following algorithms:

• QMIX: QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning
• COMA: Counterfactual Multi-Agent Policy Gradients
• VDN: Value-Decomposition Networks For Cooperative Multi-Agent Learning
• IQL: Independent Q-Learning
• QTRAN: QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning
• WQMIX: Weighted Qmix: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning
• Qatten: Qatten: A General Framework for Cooperative Multiagent Reinforcement Learning
• QPLEX: Qplex: Duplex Dueling Multi-Agent Q-Learning
• MASAC: Actor-Attention-Critic for Multi-Agent Reinforcement Learning
• SHAQ: SHAQ: Incorporating Shapley Value Theory into Multi-Agent Q-Learning
• SQDDPG: QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning
• UPDET: Updet: Universal Multi-Agent Reinforcement Learning via Policy Decoupling with Transformers
• CDS: Celebrating Diversity in Shared Multi-Agent Reinforcement Learning
• MIXRTs: MIXRTs: Toward Interpretable Multi-Agent Reinforcement Learning via Mixing Recurrent Soft Decision Trees

PyMARL is written in PyTorch and uses SMAC as its environment.

Build the Dockerfile using

cd docker
bash build.sh

Set up StarCraft II and SMAC:

bash install_sc2.sh

This will download SC2 into the 3rdparty folder and copy the maps necessary to run over.

The requirements.txt file can be used to install the necessary packages into a virtual environment (not recomended).

python3 src/main.py --config=qmix --env-config=sc2 with env_args.map_name=2s3z
python3 src/main.py --config=qmix --env-config=sc2 with env_args.map_name=5m_vs_6m gpu_id=0 t_max=2000000 epsilon_anneal_time=50000 seed=4

The config files act as defaults for an algorithm or environment.

They are all located in src/config. --config refers to the config files in src/config/algs --env-config refers to the config files in src/config/envs

All results will be stored in the Results folder.

You can save the learnt models to disk by setting save_model = True, which is set to False by default. The frequency of saving models can be adjusted using save_model_interval configuration. Models will be saved in the result directory, under the folder called models. The directory corresponding each run will contain models saved throughout the experiment, each within a folder corresponding to the number of timesteps passed since starting the learning process.

Learnt models can be loaded using the checkpoint_path parameter, after which the learning will proceed from the corresponding timestep.

save_replay option allows saving replays of models which are loaded using checkpoint_path. Once the model is successfully loaded, test_nepisode number of episodes are run on the test mode and a .SC2Replay file is saved in the Replay directory of StarCraft II. Please make sure to use the episode runner if you wish to save a replay, i.e., runner=episode. The name of the saved replay file starts with the given env_args.save_replay_prefix (map_name if empty), followed by the current timestamp.

The saved replays can be watched by double-clicking on them or using the following command:

python -m pysc2.bin.play --norender --rgb_minimap_size 0 --replay NAME.SC2Replay

Note: Replays cannot be watched using the Linux version of StarCraft II. Please use either the Mac or Windows version of the StarCraft II client.

If you use PyMARL-RLC in your research, please cite the SMAC paper and MIXRTs paper.

In BibTeX format:

@article{samvelyan19smac,
  title = {The StarCraft Multi-Agent Challenge},
  author = {Mikayel Samvelyan and Tabish Rashid and Christian Schroeder de Witt and Gregory Farquhar and Nantas Nardelli and Tim G. J. Rudner and Chia-Man Hung and Philiph H. S. Torr and Jakob Foerster and Shimon Whiteson},
  journal = {CoRR},
  volume = {abs/1902.04043},
  year = {2019},
}


@article{liu2022mixrts,
  title={MIXRTs: Toward Interpretable Multi-Agent Reinforcement Learning via Mixing Recurrent Soft Decision Trees},
  author={Liu, Zichuan and Zhu, Yuanyang and Wang, Zhi and Chen, Chunlin},
  journal = {CoRR},
  volume = {abs/2209.07225},
  year={2022}
}

Code licensed under the Apache License v2.0