RayOutOfMemoryError while doing PBA search. about pba HOT 5 CLOSED

Hey, thanks for trying out the code!

Could this be an issue with dataloading? The dataloader implemented in this repo is fairly basic and loads all examples into memory.

For the issue with ray memory and object store, I'm not too familiar unfortunately, but you may have better luck asking in the ray github: https://github.com/ray-project/ray

from pba.

My dataloader is a ray remote function calls, all data lies on disk. Whenever I need a batch I run ray remote function in parallel x times, if my batchsize is x.

from pba.

I see - unfortunately, I'm not familiar with that use case. You'll be best be able to resolve it by asking in the Ray github.

from pba.

I was able to resolve this issue, by removing memory quota specifications in ray.init() . It turns out in container environment (kubernetes) setting the memory utilization quota makes the ray quickly utilization all of the memory. If I don't set it, it goes smoothly.

from pba.

I will close this for now and open it if I face this issue again. Thanks

from pba.