Comments (3)
DI-engine doesn't set strict rules for model, but there are some conventions between model and policy. For example, if you use PPO policy, you must define actor
and critic
in your model and implement some functions like compute_actor
and compute_critic
. I think the simplest way to customize your own model is to imitate and modify the default model of a policy, like vac for PPO.
As for your case, why do you want to use pointer network, in DI-star, we only use it to output selected_units
and maintain hidden state
inside the model. And which RL policy do you want to use? Please provide more information.
from di-engine.
Thanks for your reply. Let me check that out.
As per the motivation, I want to use the pointer network to solve the Traveling salesman problem (which has a dynamic number of inputs). I have seen works that train the pointer network with the classical REINFORCE algorithm. I would like to do experiments with other more advanced RL policies.
from di-engine.
OK I get your point. And I think it is important to model a proper MDP problem if you want to use any RL algorithms. I noticed the original paper of pointer network for TSP is a kind of supervised learning, and different MDP modeling types will make essential contribution to your final implementation and performance.
For example, if you input a state and want to output the entire path per step, you can put the hidden state implementation inside of your network model. But if you want to output the following location per step, you need to modify policy to maintain hidden state, like the differences between R2D2 and DQN.
from di-engine.
Related Issues (20)
- TD3应用混合动作空间报错,AssertionError HOT 1
- how to get the ckpt file? HOT 2
- get "TypeError: __init__() got an unexpected keyword argument 'agent_obs_shape'" when running " python3 -u smac_5m6m_masac_config.py" HOT 2
- question for SMAC HOT 3
- docker内运行lunarlander_dqn_deploy失败 HOT 6
- cannot run GTrXL demo since v0.5.0 HOT 1
- BrokenPipeError: [WinError 232] 管道正在被关闭 has occurred, when running MARL algorithm QMIX in pettingzoo
- bug when running MARL algorithm Qmix in pettingzoo HOT 3
- gym_anytrading : could not broadcast input array from shape (62,) into shape (20,3) Please help!! HOT 3
- 马里奥代码咨询 HOT 1
- No example for Trading Environments HOT 1
- KeyError: 'obs_shape' HOT 2
- How to introduce other optimizers into DI-engine? HOT 2
- RuntimeError: mat1 and mat2 shapes cannot be multiplied (4x138 and 64x138) HOT 3
- How to define close function in custom environment HOT 3
- How to initialize semantic model in subprocess mode HOT 2
- About Replicating the PPO Performance in the Hopper-V3 Environment HOT 3
- Unexpected increase in memory overhead and Multi-GPU guidance in custom environments HOT 2
- Export multi-agent policies and shared training HOT 1
- Observation shape in the custom marl environment HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from di-engine.