Initial Run & Speed-up about sound-spaces HOT 3 CLOSED

Update:
pth-time is training time, which could be reduced by compiling habitat-smi with CUDA support. Moreover, is it possible to extend the PPO trainer & policy to be compatible with multi-GPUs env? Thanks!

from sound-spaces.

Hi Chaoyi,

For your first question, that message is generated when the code obtains all the scenes in the dataset and it builds a dummy dataset. Check out this line for more detail:

It's weird that your GPU training time is longer than env-time. Usually the former takes more time. I would suggest you to first check if you have the correct version of pytorch installed. And I found incorrect pytorch version (matching CUDA version) significantly slowed down the training. As for multi-GPU training, I'll point you to the DD-PPO implementation (https://github.com/facebookresearch/habitat-lab/tree/master/habitat_baselines/rl/ddppo) in Habiat, which utilizes multi-gpus and speeds up training.

from sound-spaces.

Many thanks Changan! I will have a close study on it.

from sound-spaces.