Comments (6)
could you please try this?
https://stackoverflow.com/questions/16526783/python-subprocess-too-many-open-files
Loading all files at once may not be a suitable solution for low ram settings. I will try to change it to npy dataloader this weekend.
from medsam.
from medsam.
this is wired. our node has 1TB RAM and it works. i will check it again.
from medsam.
Having set ulimit over the number of files (i.e. >161k) i do get it to run, however it eats all the memory available in the node. Hope you can see this screen shot.
from medsam.
Yes. This can bring faster training speed. There is always a trade-off.
I will provide a script for batch loading by the end of this week.
from medsam.
Hi @MrDotOne ,
I have changed the dataloader to npy dataloader. The data loading should cost less RAM now (at the cost of using more space on hard drive).
from medsam.
Related Issues (20)
- point prompt inference HOT 2
- why image data must be normalized to [0, 1], 1]"? HOT 1
- The training process of LiteMedSAM HOT 1
- About the bounding box HOT 3
- Prompt method HOT 1
- About GUI segment in WSI patches HOT 1
- 2d custom dataset supported? HOT 1
- bbox needed for inference? HOT 2
- labeled areas HOT 1
- Why only randomly select a label as a hint in the ground truth mask? HOT 1
- Is their any possibilities to add SAM2AutomaticMaskGenerator ? HOT 3
- Hi I can't open this dataset link HOT 6
- dataset HOT 1
- question about the iou loss HOT 2
- about point prompt HOT 1
- question about class_name HOT 1
- 关于二维数据集的问题 HOT 1
- MedSAM2 datasplit and inference HOT 5
- About image size HOT 1
- 关于多标签掩码的处理 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from medsam.