tesfayz / ccm_madrl_mec Goto Github PK

Hello, I have read your paper and open-source code and feel that I have benefited a lot. But as I have just started, there are still some areas that I don't understand that I would like to ask you for advice. Firstly, regarding the combinatorial optimization problem mentioned in the paper, which involves deploying agents through the madddpg algorithm on devices and having a DQN agent in the edge network. The DQN agent outputs offloading scheduling actions based on network status and device actions, but I have not observed any part about DQN in open-source code. In addition, the article mentioned using Huawei's task dataset for validation, and I did not understand how this dataset can be applied to the MADRL environment. Thank you for your reply amidst your busy schedule. Wishing you smooth research.

In line 276 of CCM_MADDPG.py, I wonder why " newactor_action_var = self.actors[agent_id](states_var[:, agent_id, :]" instead of "newactor_action_var = self.actors[agent_id](next_states_var[:, agent_id, :])" when calculating the next target actions for each agent? Hope to get your answer, thank you very much.

Dear author:
First of all, thank you for your project which has brought me a lot of inspiration. At the same time, in the part of reward function, I have a problem.
I tried to print the energy, energy penalty, time and time penalty, and I found that the energy data and time data were several orders of magnitude different. Time is ≈10^-1 and energy is ≤10^-3. Both of their weight are 0.5 and 0.5.(step10). I want to know the rationality of designing rewards in this way. I wonder if it is necessary to normalize or standardize the time delay and energy before designing the reward function.
Beside, regarding the weight of reward function, whether it is 0.5 and 0.5 or 1 and 5, are you experimenting constantly? Is there a better way to set their weights scientifically?
These are some of my questions, and I would appreciate it if you could give me some guidance.

Dear author, I am very interested in your work, may I ask how long you run an experiment?

您好，我是一名新手，我在运行run.py的时候出现 InfdexofResult = sys.argv[1] # set run runnumber for indexing results,
IndexError: list index out of range，请问是什么原因呢？

Hello author, it is stated in the paper that the master will decide whether to accept the offloading task based on the state information and actions of all client agents. However, I don't seem to have found any relevant operation in the code. Can you please indicate it?

I think your work is interesting and have tried to reproduce your work, but I have experienced overfitting many times. I noticed that you mention in your paper that this is normal, but I can't quite understand how overfitting can prove that your algorithm can outperform a benchmark algorithm if it occurs.This is because under the same environmental parameters, sometimes overfitting occurs and sometimes it is able to train and optimise normally.

Sorry to bother you, I want to ask what 'env' is.