when I run train.py there is only one GPU has utilization about pba HOT 3 OPEN

Although the code has only been tested with 1 GPU, you usually have to specify CUDA_VISIBLE_DEVICES to constrain the search.

Perhaps a newer version of Ray makes this constraint instead?

from pba.

I've added CUDA_VISIBLE_DEVICES=0,1,2,3 to the scripts。
It's no problem on search.py, because run search.py, gpu is a fractional value, and num_samples > 1 . It works with multiple GPUs on Ray.
But, I mean train.py --num_samples=1, --gpu=4, ... here gpu nums > 1, num_samples=1, it means resources_per_trial:{"gpu":4} , classification model train is too slow, it does not work on 4 GPUs, only one GPU has utilization。

So, when --num_samples=1,--gpu>1 , model could not work with multiple GPUs on Ray ?

from pba.

I see; single model parallelism across GPUs is unfortunately not supported.

from pba.