Comments (7)
ziw-liu
commented on June 25, 2024
1
Thank you @ziw-liu ! What is different when the data is on disk or scratch, since the code doesn't change. Does the data become untracked array on scratch?
It shouldn't. I suspect that some file system discrepancy covered this bug when you processed the data on an external storage server.
from microdl.
ziw-liu
commented on June 25, 2024
1
Training now works on local disk.
from microdl.
ziw-liu
commented on June 25, 2024
There seems to be an iohub warning in your logs:
WARNING:root:Skipped item at 0: invalid <class 'type'>.
WARNING:root:Skipped item at 0: invalid <class 'type'>.
WARNING:root:Skipped item at 0: invalid <class 'type'>.
Have you pushed your preprocessing code to #213?
from microdl.
Soorya19Pradeep
commented on June 25, 2024
Yes. It is not present with the run from the disk. Also I am working from the clean cop of the branch iohub_dependency without any modifications.
from microdl.
ziw-liu
commented on June 25, 2024
@Soorya19Pradeep are you on the HEAD of main for your iohub installation?
Sorry for the confusion. This bug is likely due to the custom microDL code reading and writing 'untracked arrays' that uses a different dimension separator . instead of / used in NGFF v0.4.
from microdl.
Soorya19Pradeep
commented on June 25, 2024
Thank you @ziw-liu ! What is different when the data is on disk or scratch, since the code doesn't change. Does the data become untracked array on scratch?
from microdl.
ziw-liu
commented on June 25, 2024
Fixing in fee41b3.
from microdl.
Related Issues (20)
- Config issues on gunpowder dataloading branch HOT 12
- Flatfielding computation error on gunpowder dataloading branch HOT 2
- Improve masking of fluorescence data in preprocessing HOT 2
- configs should use channel names and not channel index HOT 2
- Metadata Structure HOT 2
- z-score HOT 1
- Inference shouldn't use gunpowder HOT 5
- intensity.csv HOT 1
- Convert existing tensorflow models to pytorch HOT 3
- Does inference need its own config file? HOT 1
- data problem HOT 3
- Make model architecture compatible with deployment HOT 9
- Training error without augmentation HOT 1
- Normalization statistics don't need to be stored with additional key HOT 1
- Unexpected behavior in model batch prediction? HOT 8
- Runtime error testing lightning pipeline HOT 5
- unify preprocessing CLI with lightning CLI HOT 1
- Brain_2.5DUNet HOT 4
- Paper Figure 5. HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from microdl.