Comments (2)
I understand what you mean by non-marl distributed training of a single network, unfortunately at this moment I donβt have a working example of this. I would start by scouring the OpenAI gym and Stablebaselines docs to see if its implemented in any examples there, which would likely mean the same can be done in Mindmaker using similar methods, you would probably need to modify the source a bit. This might be what your looking for
https://github.com/Rohan138/marl-baselines3
https://github.com/HumanCompatibleAI/adversarial-policies/blob/baa359420641b721aa8132d289c3318edc45ec74/src/aprl/envs/multi_agent.py
from mindmaker.
I am working on the same issue. Parallel training is the only missing feature for this plugin before we can do serious research with, compared to using Unity ML-agents or OpenAI Gym.
The problem seems that we need to get the action/observation tuple in UE first, then dispatch it to each agent there. A lot of implementations automatically dispatch the data to each environment, which means we only get a single element in UE, which isn't what we want.
There are a few ways I have tried so far, albeit unsuccessfully. If you have any ideas on how to implement it I'll be interested.
- Implementing a vectorized environment from StableBaselines3 library
Explanation here: https://stable-baselines3.readthedocs.io/en/master/guide/vec_envs.html
So I end up creating different environments from the DummyVecEnv following the guidelines:
# DummyVecEnv
def make_env(rank, seed = 0):
env = UnrealEnvWrap()
env.seed(rank + seed)
return env
env = DummyVecEnv([make_env(i) for i in range(n_agents)])
However I get the following error :
File "C:\Users\fioshirk\Anaconda3\envs\Mindmaker\lib\threading.py", line 980, in _bootstrap_inner
self.run()
File "C:\Users\fioshirk\Anaconda3\envs\Mindmaker\lib\threading.py", line 917, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\fioshirk\Anaconda3\envs\Mindmaker\lib\site-packages\socketio\server.py", line 731, in _handle_event_internal
r = server._trigger_event(data[0], namespace, sid, *data[1:])
File "C:\Users\fioshirk\Anaconda3\envs\Mindmaker\lib\site-packages\socketio\server.py", line 756, in _trigger_event
return self.handlers[namespace]event
File "C:\Users\fioshirk\Desktop\MMS2\25.py", line 243, in receive
env = DummyVecEnv([make_env(i) for i in range(n_agents)])
File "C:\Users\fioshirk\Anaconda3\envs\Mindmaker\lib\site-packages\stable_baselines3\common\vec_env\dummy_vec_env.py", line 26, in init
self.envs = [fn() for fn in env_fns]
File "C:\Users\fioshirk\Anaconda3\envs\Mindmaker\lib\site-packages\stable_baselines3\common\vec_env\dummy_vec_env.py", line 26, in
self.envs = [fn() for fn in env_fnsTypeError: 'UnrealEnvWrap' object is not callable
- Implementing a custom vectorized environment
Instead of relying on the SB vectorized environment, we can create our own. It would be an environment that can loop through a tuple of environments at each step.
class VectorizedEnvironment(gym.Env):
def __init__(self, make_env, n):
self.envs = tuple(make_env() for _ in range(n))
print("Environments in Vectorized Environment: ",self.envs)
[...]
def step(self, action):
global observations
global reward
global UEreward
global UEdone
global obsflag
obsflag = 0
# Loop through each environment:
for env, a in zip(self.envs, action):
print("Step for environment number ",a)
# send actions to UE as they are chosen by the RL algorithm
However it seems impossible to pass a tuple for the observation space, the model requires a gym space instead (for example a box). So not sure how to tackle this one either.
I'll have a look at the examples Krumiaa sent and perhaps it would work better.
from mindmaker.
Related Issues (20)
- Error when launching the example maps HOT 4
- Can the low and high ends of the observation shape vary?
- Unreal Engine crashes upon compiling or running anything involving the SocketIO client HOT 1
- How to properly reset my Agent when he does something fatal? HOT 5
- UE5: MindMaker (DL single node) gives (KeyError: '_init_setup_model') on play HOT 28
- Mindmaker is returning the same action for every receiveAction call during Evaluation phase. HOT 22
- TD3 Action noise
- How to integrate DRL into an already existing unreal 4.27 project HOT 1
- Terminating episode early HOT 4
- MARL disconnection from Unreal Engine after a while HOT 4
- ImportError: DLL load failed HOT 1
- Error training with PPO2 Full tensorbord logs = True HOT 1
- UE5 Plugin does not load learning engine HOT 5
- What does mindmaker showing the error while launching it? HOT 3
- PermissionError: [Errno 13] Permission denied: 'C:\\Users\\user\\AppData\\Roaming\\PPOCart' HOT 4
- How to include learned .pth files in a build instead of a PIE run
- problem building Socketio for UE 5.1
- problem "local variable 'policy_kwargsval' referenced before assignment"
- dont understand how the agent visualization works.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mindmaker.