liyang619 / cole-platform Goto Github PK
View Code? Open in Web Editor NEWOvercooked human-AI experiment platform
License: MIT License
Overcooked human-AI experiment platform
License: MIT License
Hi, I am interested in your implementation of fictitious co-play in this repo, but I had a hard time finding it. Could you leave a pointer for that?
Or is it this one? (Does not look like the algorithm described in the FCP paper though)
https://github.com/liyang619/COLE-Platform/blob/COLE_training/human_aware_rl/pbt_original/fcp.py
I am trying to pair your reproduced baselines (downloaded from Google Drive) with my own human proxy model trained in the new Overcooked-AI environment (with old dynamics).
To make them compatible I first use the old lossless_state_encoding
function for the baselines (only for the baselines model because my HP is trained in the new env).
Also, the env requires tensorflow 2
but your models are trained using tensorflow 1
, so I load your models through the following code:
def get_model_policy_from_saved_model(save_dir, sim_threads=30):
"""Get a policy function from a saved model"""
predictor = tf.saved_model.load(save_dir)
step_fn = lambda obs: predictor.signatures["serving_default"](tf.convert_to_tensor(obs, dtype=tf.float32))["action_probs"]
return get_model_policy(step_fn, sim_threads)
However this would raise warnings like:
WARNING:tensorflow:Unable to create a python object for variable <tf.Variable 'agent0/ppo2_model/pi/conv_0/kernel:0' shape=(3, 3, 25, 25) dtype=float32_ref> because it is a reference variable. It may not be visible to training APIs. If this is a problem, consider rebuilding the SavedModel after running tf.compat.v1.enable_resource_variables().
The code to pair the MEP baseline with my human proxy model is as follows:
def evaluate_hp_mep(hp_model_path, mep_model_path, layout, order=0):
hp_model, hp_params = load_bc_model(hp_model_path)
hp_policy = BehaviorCloningPolicy.from_model(
hp_model, hp_params, stochastic=True
)
# print(hp_params)
base_ae = _get_base_ae(hp_params)
base_env = base_ae.env
hp_agent = RlLibAgent(hp_policy, order, base_env.featurize_state_mdp)
mep_agent = get_agent_from_saved_model(mep_model_path, sim_threads=30)
ae = AgentEvaluator.from_layout_name(
mdp_params={"layout_name": layout, "old_dynamics": True},
env_params={"horizon": 400},
)
if order == 0:
ap = AgentPair(hp_agent, mep_agent)
else:
ap = AgentPair(mep_agent, hp_agent)
result = ae.evaluate_agent_pair(ap, 5, 400)
return result, result["ep_returns"]
But the evaluation results are quite unsatisfying. Am I missing some steps or the warnings do matter?
Unable to correctly load the index page on the main branch, which blocks subsequent operations.
Flask reports a "404 Not Found" error for the missing demo.js
:
"GET /static/js/demo/demo.js HTTP/1.1" 404 -
This occurs because the demo.js
file is absent in the main branch.
The demo.js
file from the human-human/llm
branch located at overcookedgym/overcooked-flask/static/js/demo/demo.js
has been copied to the corresponding location on the main branch.
After fixing the missing demo.js
file, a KeyError occurs on the playtesting page with the message:
"POST /bc/predict/ HTTP/1.1" 500 -
Traceback (most recent call last):
...
File "overcookedgym/overcooked-flask/app.py", line 165, in predict
data_json["npc_index"],
KeyError: 'npc_index'
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.