Hello. It seems like I am experiencing the same problem as the thread above #6
The latest version of Ray has changed the Model.py file to ray.rllib.models.ModelV2.py. In response, I tried changing the code for smac.examples.rllib.model.py as shown in the code attached below.

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import tensorflow as tf

from ray.rllib.models import modelv2 #changed model to modelv2
from ray.rllib.models.tf.misc import normc_initializer


class MaskedActionsModel(modelv2): #also changed model to modelv2
    def _build_layers_v2(self, input_dict, num_outputs, options):
        action_mask = input_dict["obs"]["action_mask"]
        if num_outputs != action_mask.shape[1].value:
            raise ValueError(
                "This model assumes num outputs is equal to max avail actions",
                num_outputs,
                action_mask,
            )

However, this raises another error.

"TypeError: module() takes at most 2 arguments (3 given)".
Is there a fix for this?

In the capability wrapper

The rendering function does not take the mode argument and thus we cannot render in rgb_arrays environments that are using the capability config

I was just wondering if the results in the paper are using 4.10 or older versions to be consistent with the original smac results?

I'm trying to run SMACv2 baseline QMIX in smacv2/RUNNING_EXPERIMENTS.md. However, I keep running into this error:

Hello from the pygame community. https://www.pygame.org/contribute.html
[DEBUG 20:07:32] git.util Failed checking if running in CYGWIN due to: FileNotFoundError(2, 'No such file or directory')
[DEBUG 20:07:32] git.cmd Popen(['git', 'diff', '--cached', '--abbrev=40', '--full-index', '--raw'], cwd=/source, stdin=None, shell=False, universal_newlines=False)
Traceback (most recent call last):
  File "src/main.py", line 19, in <module>
    ex = Experiment("pymarl")
  File "/opt/conda/lib/python3.8/site-packages/sacred/experiment.py", line 119, in __init__
    super().__init__(
  File "/opt/conda/lib/python3.8/site-packages/sacred/ingredient.py", line 75, in __init__
    ) = gather_sources_and_dependencies(
  File "/opt/conda/lib/python3.8/site-packages/sacred/dependencies.py", line 728, in gather_sources_and_dependencies
    experiment_path, main = get_main_file(globs, save_git_info)
  File "/opt/conda/lib/python3.8/site-packages/sacred/dependencies.py", line 599, in get_main_file
    main = Source.create(globs.get("__file__"), save_git_info)
  File "/opt/conda/lib/python3.8/site-packages/sacred/dependencies.py", line 460, in create
    repo, commit, is_dirty = get_commit_if_possible(main_file, save_git_info)
  File "/opt/conda/lib/python3.8/site-packages/sacred/dependencies.py", line 440, in get_commit_if_possible
    is_dirty = repo.is_dirty()
  File "/opt/conda/lib/python3.8/site-packages/git/repo/base.py", line 820, in is_dirty
    if osp.isfile(self.index.path) and len(self.git.diff("--cached", *default_args)):
  File "/opt/conda/lib/python3.8/site-packages/git/cmd.py", line 736, in <lambda>
    return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/git/cmd.py", line 1316, in _call_process
    return self.execute(call, **exec_kwargs)
  File "/opt/conda/lib/python3.8/site-packages/git/cmd.py", line 1111, in execute
    raise GitCommandError(redacted_command, status, stderr_value, stdout_value)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(129)
  cmdline: git diff --cached --abbrev=40 --full-index --raw
  stderr: 'Not a git repository
To compare two paths outside a working tree:
usage: git diff [--no-index] <path> <path>'

I've tried a few things but I'm not sure how to solve this. Is this a conda error or a git error? Any help on solving this would be much appreciated! Thank you.

when I try to replay the SC2Replay file in the StarCraft II client, I find a error " can not open the map".
How to fix it?

First of all, thank you very much for your contribution. I tried to use the code from: https://github.com/benellis3/pymarl2 to reproduce the results in the paper with following commond: python3 src/main.py --config=qmix --env-config=sc2_gen_protoss with env_args.capability_config.n_units=5 env_args.capability_config.start_positions.n_enemies=5

However, in protoss_ 5_ vs_5, after running 10M steps, the final test win rate is ~20%, The winning rate of protoss_10_ vs_10 for 10M steps is ~15%, which is far lower than the results in the paper. Could you please tell me if I missed any details? Or am I using Open Loop mode?

I am a beginner in multi-agent reinforcement learning. Thank you for your contributions to multi-agent reinforcement learning. I encountered this error while using smacv2. It should be a simple error of missing function, but I can't seem to find this function in starcraft2.py.

Process Process-1:
Traceback (most recent call last):
File "D:\anaconda3\lib\multiprocessing\process.py", line 314, in _bootstrap
self.run()
File "D:\anaconda3\lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "D:\code\mappo-master\onpolicy\envs\env_wrappers.py", line 316, in shareworker
ob, s_ob, available_actions = env.reset()
File "D:\code\mappo-master\onpolicy\envs\starcraft2\SMACv2.py", line 25, in reset
obs, state = super().reset()
File "C:\Users\31315\smacv2\smacv2\env\starcraft2\wrapper.py", line 33, in reset
return self.env.reset(reset_config)
File "C:\Users\31315\smacv2\smacv2\env\starcraft2\starcraft2.py", line 582, in reset
return self.get_obs(), self.get_state()
File "C:\Users\31315\smacv2\smacv2\env\starcraft2\starcraft2.py", line 1728, in get_obs
agents_obs[agent] = self.get_obs_agent(
File "C:\Users\31315\smacv2\smacv2\env\starcraft2\starcraft2.py", line 1510, in get_obs_agent
true_avail_actions = self.get_true_avail_agent_actions(agent_id)
AttributeError: 'StarCraft2Env' object has no attribute 'get_true_avail_agent_actions'

I tried running the rllib example and ran into the following error

ImportError: cannot import name 'Model' from 'ray.rllib.models' ~/miniconda3/envs/smac/lib/python3.9/site-packages/ray/rllib/models/__init__.py)

Ray/rllib version tried: 2.0.0/1.8.0/1.10.0.

Is there any specific version of ray/rllib to install? If so, please add it to the documentation or requiremetns.txt file.

Thanks
Kinal Mehta

Hi,

I'm wondering if it's possible to share the weights of the learned policies (MAPPO and QMIX). It's helpful for the adversarial MARL community. It will help us to focus on prototyping adversarial algorithms (black-box setting) rather than spending a lot of time trying to train a policy from scratch.

Thanks.

I use the code provided in the README.md to create a new testing environment, and forced seed=1 when constructing the StarCraft2Env.
However, the results given by each run are different. How should I set the seed to effectively reproduce the previous results?

Hi, I just wanted to say this is an awesome project and I'm excited to try it out.

I installed via the instructions but it doesn't install pettingzoo, even though there are examples using it. Obviously users can just install it themselves but I think it should probably be listed as an extra. As a note, there is a new release upcoming in the next few weeks so it could wait until that, or just list pettingzoo>=1.22.4 (most recent version).

If you need any help working out kinks due to different versions feel free to ask, there were some breaking changes in the past 6 months or so, so if you previously used an older version like 1.15.0 it requires a bit of updates to fix. The previous API returned done in the step() function, whereas the new one returns truncated and terminated (matching gymnasium). There is a migration guide for gymnasium explaining the changes further, the steps should be basically the same (we're working on making resources for updating old PettingZoo repositories as well): https://gymnasium.farama.org/content/migration-guide/

Hi, thanks for providing SMAC v2!!

I have one questions comparing with SMAC v1.

Can we use SMAC 1 env in SMAC v2? For example, I want to run 3m, 2s_vs_3z, etc in SMAC v1.

I tried to learn SMAC v1 maps in SMAC v2, so I run env-config with sc2.py in SMAC V1 github.

But It dosen't start training, and giving same message as below.

How can I run smac 1 maps(3m, 2s_vs_3z, etc) in smac v2?

Thanks!

Hi,
I just installed SMAC_V2 and am experimenting with the sample code provided in the Readme. Generally, everything is working well but I see that...

If you uncomment

env.render()

an error is thrown:

Traceback (most recent call last):
  File "/Users/justinrodriguez/git/explainable_starcraft/test_driver.py", line 87, in <module>
    main()
  File "/Users/justinrodriguez/git/explainable_starcraft/test_driver.py", line 64, in main
    env.render()  # Uncomment for rendering
    ^^^^^^^^^^^^
  File "/Users/justinrodriguez/git/explainable_starcraft/.venv/lib/python3.12/site-packages/smacv2/env/starcraft2/wrapper.py", line 82, in render
    return self.env.render(mode=mode)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/justinrodriguez/git/explainable_starcraft/.venv/lib/python3.12/site-packages/smacv2/env/starcraft2/starcraft2.py", line 2285, in render
    from smacv2.env.starcraft2.render import StarCraft2Renderer
  File "/Users/justinrodriguez/git/explainable_starcraft/.venv/lib/python3.12/site-packages/smacv2/env/starcraft2/render.py", line 15, in <module>
    from pysc2.lib.renderer_human import _Surface
  File "/Users/justinrodriguez/git/explainable_starcraft/.venv/lib/python3.12/site-packages/pysc2/lib/renderer_human.py", line 35, in <module>
    from pysc2.lib import features
  File "/Users/justinrodriguez/git/explainable_starcraft/.venv/lib/python3.12/site-packages/pysc2/lib/features.py", line 350, in <module>
    SCREEN_FEATURES = ScreenFeatures(
                      ^^^^^^^^^^^^^^^
  File "/Users/justinrodriguez/git/explainable_starcraft/.venv/lib/python3.12/site-packages/pysc2/lib/features.py", line 323, in __new__
    palette=palette(scale) if callable(palette) else palette,
            ^^^^^^^^^^^^^^
  File "/Users/justinrodriguez/git/explainable_starcraft/.venv/lib/python3.12/site-packages/pysc2/lib/colors.py", line 213, in unit_type
    return categorical(static_data.UNIT_TYPES, scale)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/justinrodriguez/git/explainable_starcraft/.venv/lib/python3.12/site-packages/pysc2/lib/colors.py", line 224, in categorical
    palette = shuffled_hue(palette_size)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/justinrodriguez/git/explainable_starcraft/.venv/lib/python3.12/site-packages/pysc2/lib/colors.py", line 121, in shuffled_hue
    random.shuffle(palette, lambda: 0.5)  # Return a fixed shuffle
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Random.shuffle() takes 2 positional arguments but 3 were given
INFO:absl:Shutdown gracefully.
INFO:absl:Shutdown with return code: -15

This stems from the lambda function embedded here:

def shuffled_hue(scale):
  palette = list(smooth_hue_palette(scale))
  random.shuffle(palette, lambda: 0.5)  # Return a fixed shuffle
  return numpy.array(palette)

If you remove the lambda function in random.shuffle(), the problem goes away and the game is able to render.

FWIW I'm running in python 3.12.2

In smacv2
/RUNNING_EXPERIMENTS.md step 0 is to set the WANDB_API_KEY_FILE environment variable as mentioned in the introduction. However, in the Introduction, there is some missing text on how to do this. Could you please help me set that up?

Thanks!

I downloaded the original smac maps and 32x32_flat.SC2Map is not included among them. Where can I find it?

Hello,

Based on the state-of-the-art algorithm, the reality is that the winning rate is close to 1 in many maps. Is the author interested in further expanding the function of SMAC to realize the battle between two models obtained by different algorithms? I think this can break through the upper limit of the difficulty of SC2's built-in computer, so as to keep SMAC alive forever.

the compatible petting-zoo version for SMACv2 should be 1.8.1 @benellis3

https://github.com/oxwhirl/smacv2/blob/main/smacv2/env/starcraft2/starcraft2.py#L490

this line will crash in modern numpy versions.

I suggest to either update this or fix the requirements

I am trying to understand how the mean_test_win_rate is obtained.
From my understanding, the environment keeps track of battles played and battles won, and return win_rate when calling the get_stats() method as won/played. As I understand it, the win rate will then accumulate over test episodes. For example, test iteration 1 consists of 32 episodes, and the agents win 4 of them. Win rate is then 4/32 = 0.125. Test iteration 2 consists of another 32 episodes, and the agents win 6 of them. The win rate then becomes 10/64 = 0.15625. Is my understanding of this correct, or where am I going wrong?

I am fairly certain that I am going wrong somewhere, because if you look at the QMIX results from the provided .pkl file, specifically "protoss_5_vs_5" "QMIX" seed number "2", then win rates are 0, 0, 0.125, 0 for the first 4 logging steps. This does not make sense, as if the agents win no episodes in the next training run, the win rate will decrease but it can never go back down to 0.

Could someone please provide some clarity on this?

Hi, I am interested in your SMACv2 that it makes the original SMAC more challenging but I encountered some problems. I tried to save a replay to examine if the agents really learn to ‘snap’ their cone, however, no lines are displayed like the ones in your README file. Then I added the render_fovs() function in the save_replay() function but it was still not working. Is there a way to correctly render them? Thanks!

It seems that the paper is originally posted on the openreview, but I can't find it more.

Hi, the paper has mentioned 15 scenarios for smacv2, and how to reimplement these scenarios by modifyinng the proposed yaml?
Thanks a lot~

Hello, firstly, thank you for the contribution about this whole simulation environment.

I wanted to integrate SMACv2 with PyMARL, and luckly found the SMACv2 version of PyMARL, PyMARL2
https://github.com/benellis3/pymarl2 (@benellis3)

I tried using MAC OX but faced following error

File "/anaconda3/envs/smac/lib/python3.9/site-packages/pysc2/maps/lib.py", line 122, in get_maps
raise DuplicateMapError("Duplicate map found: " + map_name)
pysc2.maps.lib.DuplicateMapError: Duplicate map found: 3m

After I delete 2 lines as below, it seems working, but is it a good solution for this error?

https://github.com/deepmind/pysc2/blob/0df53d38c153972f1e368572ba65b1442a0fd41f/pysc2/maps/lib.py#L121
https://github.com/deepmind/pysc2/blob/0df53d38c153972f1e368572ba65b1442a0fd41f/pysc2/maps/lib.py#L122

Thank you.

Being the new guy here, as I read through the documentation I see that several different scenarios are defined. Unfortunately I don't see anything that clarifies how to specify which scenario you wish to run. As a suggestion, could the documentation be updated to explain this?

Thanks!

Hi everyone,
First of all, thanks for both these amazing library, they are really an amazing contribution to MARL experimentation!
Now, I was trying to get both SMAC and SMACv2 into the same pymarl codebase. So I installed both, added the SMAC_Maps from SMAC into the correct folder $HOME/StarCraftII/Maps and modified src.envs.__init__.py as follow:

from smac.env import StarCraft2Env
from smacv2.env import StarCraft2Env as StarCraft2v2Env

...

REGISTRY["sc2"] = partial(env_fn, env=StarCraft2Env)
REGISTRY["sc2-v2"] = partial(env_fn, env=StarCraft2v2Env)

However, when I try to launch an algorithm on any of the two, e.g.

python3 src/main.py --config=qmix --env-config=sc2

I keep getting this error (during the last attempt is was reported twice, but the number of lines vary depending on what I try)

pysc2.maps.lib.DuplicateMapError: Duplicate map found: 3m
pysc2.maps.lib.DuplicateMapError: Duplicate map found: 3m

If I simply uninstall smacv2 however everything works just fine. So, is there a way to have both libraries to work together?

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from os import replace

from smac.env import StarCraft2Env
import numpy as np
from absl import logging
import time

from smac.env.starcraft2.wrapper import StarCraftCapabilityEnvWrapper

logging.set_verbosity(logging.DEBUG)

def main():

    distribution_config = {
        "n_units": 5,
        "team_gen": {
            "dist_type": "weighted_teams",
            "unit_types": ["marine", "marauder", "medivac"],
            "exception_unit_types": ["medivac"],
            "weights": [0.45, 0.45, 0.1],
            "observe": True,
        },
        "start_positions": {
            "dist_type": "surrounded_and_reflect",
            "p": 0.5,
            "n_enemies": 5,
            "map_x": 32,
            "map_y": 32,
        },
    }
    env = StarCraftCapabilityEnvWrapper(
        capability_config=distribution_config,
        map_name="10gen_terran",
        debug=True,
        conic_fov=False,
        obs_own_pos=True,
        use_unit_ranges=True,
        min_attack_range=2,
    )

    env_info = env.get_env_info()

    n_actions = env_info["n_actions"]
    n_agents = env_info["n_agents"]

    n_episodes = 10

    print("Training episodes")
    for e in range(n_episodes):
        env.reset()
        terminated = False
        episode_reward = 0

        while not terminated:
            obs = env.get_obs()
            state = env.get_state()
            # env.render()  # Uncomment for rendering

            actions = []
            for agent_id in range(n_agents):
                avail_actions = env.get_avail_agent_actions(agent_id)
                avail_actions_ind = np.nonzero(avail_actions)[0]
                action = np.random.choice(avail_actions_ind)
                actions.append(action)

            reward, terminated, _ = env.step(actions)
            time.sleep(0.15)
            episode_reward += reward
        print("Total reward in episode {} = {}".format(e, episode_reward))

if __name__ == "__main__":
    main()

Bugs:

• import say smac but shouldn't it be smacv2?

  File "/Users/matbet/PycharmProjects/rl/torchrl/envs/libs/smac_try.py", line 77, in <module>
    main()
  File "/Users/matbet/PycharmProjects/rl/torchrl/envs/libs/smac_try.py", line 35, in main
    env = StarCraftCapabilityEnvWrapper(
  File "/Users/matbet/miniconda3/envs/torchrl/lib/python3.9/site-packages/smacv2/env/starcraft2/wrapper.py", line 10, in __init__
    self._parse_distribution_config()
  File "/Users/matbet/miniconda3/envs/torchrl/lib/python3.9/site-packages/smacv2/env/starcraft2/wrapper.py", line 24, in _parse_distribution_config
    config["n_enemies"] = self.distribution_config["n_enemies"]
KeyError: 'n_enemies'

n_enemies in the config seems to be in the wrong place?

I want to save the map files of different n_units and unit_types so that I can eval my model on these maps. How can I generate plenty of .SC2Map files with smacv2?

Hello, thank you for your contribution of rllib version.
When I run run_ppo.py and run_qmix.py, there is a Value Error as below

(RolloutWorker pid=3950203) File "../python3.9/site-packages/ray/rllib/evaluation/rollout_worker.py", line 682, in init
(RolloutWorker pid=3950203) self.policy_dict, self.is_policy_to_train = self.config.get_multi_agent_setup(
(RolloutWorker pid=3950203) File "../python3.9/site-packages/ray/rllib/algorithms/algorithm_config.py", line 2728, in get_multi_agent_setup
(RolloutWorker pid=3950203) raise ValueError(
(RolloutWorker pid=3950203) ValueError: observation_space not provided in PolicySpec for default_policy and env does not have an observation space OR no spaces received from other workers' env(s) OR no observation_space specified in config!
ValueError: observation_space not provided in PolicySpec for default_policy and env does not have an observation space OR no spaces received from other workers' env(s) OR no observation_space specified in config!

while my tensorflow version is 2.12, ray version is 2.4.0, python version is 3.9

Waiting for you answer, thx

Hi, I find some problems in your readme, which may confuse us.

1. In the "capability config" and code example", the item ""n_enemies": 5 " shouldn't be under strat_positions, it is should be closed to "n_unit".
2. it imports from smac in the example code, you should import from smacv2?

Besides, which version of pettingzoo do you use? I cannot run the example code of pettingzoo, which raises error "ImportError: cannot import name 'from_parallel_wrapper' from 'pettingzoo.utils.conversions'".

Hi, I am interested in your SMACv2 that it makes the original SMAC more challenging but I encountered some problems.
When I tried to run on some original maps like "3m" to use the stochastic spawn location and sectional observation, I found that my program was always stuck in a while loop in the function self._kill_all_units(). And always receive:

RequestQuit command received.
Closing Application...
DataHandler: unable to parse websocket frame.
CloseHandler: 127.0.0.1:46776 disconnected
Starting up...
Startup Phase 1 complete
[INFO 15:27:44] absl Shutdown gracefully.
[INFO 15:27:44] absl Shutdown with return code: -15

In smac/env/starcraft2/starcraft2.py

def _restart(self):
    """Restart the environment by killing all units on the map.
    There is a trigger in the SC2Map file, which restarts the
    episode when there are no units left.
    """
    try:
        self._kill_all_units()
    except (protocol.ProtocolError, protocol.ConnectionError):
        self.full_restart()

def _kill_all_units(self):
    """Kill all units on the map. Steps controller and so can throw
    exceptions"""
    units = [unit.tag for unit in self._obs.observation.raw_data.units]
    self._kill_units(units)
    # check the units are dead
    units = len(self._obs.observation.raw_data.units)
    while len(self._obs.observation.raw_data.units) > 0:
        self._controller.step(2)
        self._obs = self._controller.observe()

On map "3m", "len(self._obs.observation.raw_data.units)" is always 6. Is it because that it does not further kill units in the while loop? After I replace the _restart and _kill_all_units functions with the code from the original smac, everything is fine.

When I try to use 'python -m smacv2.examples.random_agents' to test smacv2, it says 'StarCraft2Env' object has no attribute 'state_feature_names'.

Hi,

I see the 4 main contributions of SMACv2 are as follows

• random team compositions
• random start positions
• conic fov
• conic attack/shoot

I want to disable follows

• random team compositions (only increase stochasticity by start positions)
• conic fov (attack should follow conic constraints, but fov should be like SMACv1)

This would help reproduce the paper's ablation studies.
I'm unable to configure the above described behaviour. Could you please direct me to the relevant reference? It'll be great if there is an example of this.

Thanks
Kinal