train error with Google Drive data about medsam HOT 6 CLOSED

could you please try this?

https://stackoverflow.com/questions/16526783/python-subprocess-too-many-open-files

Loading all files at once may not be a suitable solution for low ram settings. I will try to change it to npy dataloader this weekend.

from medsam.

from medsam.

this is wired. our node has 1TB RAM and it works. i will check it again.

from medsam.

Having set ulimit over the number of files (i.e. >161k) i do get it to run, however it eats all the memory available in the node. Hope you can see this screen shot.

from medsam.

Yes. This can bring faster training speed. There is always a trade-off.

I will provide a script for batch loading by the end of this week.

from medsam.

Hi @MrDotOne ,

I have changed the dataloader to npy dataloader. The data loading should cost less RAM now (at the cost of using more space on hard drive).

from medsam.