Comments (14)
bpinsard
commented on August 16, 2024
1
The defacing was skipped on that images, which is a mistake. And this image still needs defacing as the FoV covers a good chunk of the teeth which are identifiable features (I have no knowledge about forensic odontology but it's better to be on the safe side). Will fix it ASAP.
from cneuromod_docs.
jcohenadad
commented on August 16, 2024
1
julien-macbook:~/temp/neuromod/anat $ datalad update --merge
[INFO ] Fetching updates for Dataset(/Users/julien/temp/neuromod/anat)
merge(ok): . (dataset) [Merged origin/main]
update.annex_merge(ok): . (dataset) [Merged annex branch]
update(ok): . (dataset)
action summary:
merge (ok: 1)
update (ok: 1)
update.annex_merge (ok: 1)
julien-macbook:~/temp/neuromod/anat $ git tag
0.10.0
0.10.1
2020-alpha2
julien-macbook:~/temp/neuromod/anat $ datalad get sub-01/ses-001/
get(error): sub-01/ses-001/anat/sub-01_ses-001_acq-axial_localizer.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(error): sub-01/ses-001/anat/sub-01_ses-001_acq-coronal_bp-cspine_localizer.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(error): sub-01/ses-001/anat/sub-01_ses-001_acq-coronal_localizer.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(error): sub-01/ses-001/anat/sub-01_ses-001_acq-sagittal_bp-cspine_localizer.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(error): sub-01/ses-001/anat/sub-01_ses-001_acq-sagittal_localizer.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(ok): sub-01/ses-001/anat/sub-01_ses-001_bp-cspine_flip-1_mt-off_MTS.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-001/anat/sub-01_ses-001_bp-cspine_flip-1_mt-on_MTS.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-001/anat/sub-01_ses-001_bp-cspine_flip-2_mt-off_MTS.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-001/anat/sub-01_ses-001_flip-1_mt-off_MTS.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-001/anat/sub-01_ses-001_flip-1_mt-on_MTS.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-001/anat/sub-01_ses-001_flip-2_mt-off_MTS.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-001/anat/sub-01_ses-001_rec-dis2d_flip-1_mt-off_MTS.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-001/anat/sub-01_ses-001_rec-dis2d_flip-1_mt-on_MTS.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-001/anat/sub-01_ses-001_rec-dis2d_flip-2_mt-off_MTS.nii.gz (file) [from cneuromod.anat.mri...]
get(error): sub-01/ses-001/anat/sub-01_ses-001_rec-norm_mod-T1w_defacemask.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(error): sub-01/ses-001/sub-01_ses-001_scans.tsv (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(impossible): sub-01/ses-001 (directory) [could not get some content in /Users/julien/temp/neuromod/anat/sub-01/ses-001 ['/Users/julien/temp/neuromod/anat/sub-01/ses-001/anat/sub-01_ses-001_acq-axial_localizer.nii.gz', '/Users/julien/temp/neuromod/anat/sub-01/ses-001/anat/sub-01_ses-001_acq-coronal_bp-cspine_localizer.nii.gz', '/Users/julien/temp/neuromod/anat/sub-01/ses-001/anat/sub-01_ses-001_acq-coronal_localizer.nii.gz', '/Users/julien/temp/neuromod/anat/sub-01/ses-001/anat/sub-01_ses-001_acq-sagittal_bp-cspine_localizer.nii.gz', '/Users/julien/temp/neuromod/anat/sub-01/ses-001/anat/sub-01_ses-001_acq-sagittal_localizer.nii.gz', '/Users/julien/temp/neuromod/anat/sub-01/ses-001/anat/sub-01_ses-001_rec-norm_mod-T1w_defacemask.nii.gz', '/Users/julien/temp/neuromod/anat/sub-01/ses-001/sub-01_ses-001_scans.tsv']]
action summary:
get (error: 7, impossible: 1, ok: 9)looks good! thank you!
from cneuromod_docs.
bpinsard
commented on August 16, 2024
1
Oops, both should be fixed now. Thanks for all the troubleshooting
from cneuromod_docs.
jcohenadad
commented on August 16, 2024
1
LGTM! I'll reopen if I spot other issues. Thanks a bunch!
Tagging @mathieuboudreau so he can also try brain processing on his end
from cneuromod_docs.
jcohenadad
commented on August 16, 2024
Is it possible this issue is related to the change in file names?
from cneuromod_docs.
bpinsard
commented on August 16, 2024
This is due to some files being flagged as sensitive (non-defaced or sensitive info): mainly the localizer series and the _scans.tsv (containing the real datetime of acquisition). I see how it is disturbing having these errors when downloading files.
I will investigate if git-annex allows a default filter when getting files to exclude these.
Another option would be to deface the localizer and change the datetime to fake dates (still keeping the same intervals for longitudinal analysis requiring it) which is something we already planned for the functional data.
If not, we might need to remove these from the release (still keeping them in the datalad history).
from cneuromod_docs.
bpinsard
commented on August 16, 2024
However the defacemask should be downloadable. Will look into it.
from cneuromod_docs.
jcohenadad
commented on August 16, 2024
This is due to some files being flagged as sensitive (non-defaced or sensitive info): mainly the localizer series and the _scans.tsv (containing the real datetime of acquisition). I see how it is disturbing having these errors when downloading files.
maybe this explanation is valid for some files (eg localizer), but this file for example does not show the face (axial acquisition in the neck):
get(error): sub-01/ses-001/anat/sub-01_ses-001_bp-cspine_flip-1_mt-on_MTS.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
from cneuromod_docs.
bpinsard
commented on August 16, 2024
This should be fixed, you can datalad update --merge and check that you are on tag 0.10.1 .
Now only the localizers and _scans.tsv should be inaccessible.
from cneuromod_docs.
jcohenadad
commented on August 16, 2024
Reopening because looking at another session, some scans that should be there are not there:
get(error): sub-01/ses-002/anat/sub-01_ses-002_bp-cspine_T2w.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(error): sub-01/ses-002/anat/sub-01_ses-002_inv-1_MP2RAGE.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(error): sub-01/ses-002/anat/sub-01_ses-002_inv-2_MP2RAGE.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(error): sub-01/ses-002/anat/sub-01_ses-002_rec-dis2d_UNIT1.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(error): sub-01/ses-002/anat/sub-01_ses-002_rec-dis2d_inv-1_MP2RAGE.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(error): sub-01/ses-002/anat/sub-01_ses-002_rec-dis2d_inv-2_MP2RAGE.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
Terminal output
julien-macbook:~/temp/neuromod/anat $ datalad get sub-01/ses-002/
get(ok): sub-01/ses-002/anat/sub-01_ses-002_T1w.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-002/anat/sub-01_ses-002_T2w.nii.gz (file) [from cneuromod.anat.mri...]
get(error): sub-01/ses-002/anat/sub-01_ses-002_UNIT1.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(error): sub-01/ses-002/anat/sub-01_ses-002_acq-axial_bp-cspine_localizer.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(error): sub-01/ses-002/anat/sub-01_ses-002_acq-axial_localizer.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(error): sub-01/ses-002/anat/sub-01_ses-002_acq-axial_rec-dis2d_localizer.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(error): sub-01/ses-002/anat/sub-01_ses-002_acq-coronal_bp-cspine_localizer.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(error): sub-01/ses-002/anat/sub-01_ses-002_acq-coronal_localizer.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(error): sub-01/ses-002/anat/sub-01_ses-002_acq-coronal_rec-dis2d_localizer.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(error): sub-01/ses-002/anat/sub-01_ses-002_acq-sagittal_bp-cspine_localizer.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(error): sub-01/ses-002/anat/sub-01_ses-002_acq-sagittal_localizer.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(error): sub-01/ses-002/anat/sub-01_ses-002_acq-sagittal_rec-dis2d_localizer.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(ok): sub-01/ses-002/anat/sub-01_ses-002_bp-cspine_T2starw.nii.gz (file) [from cneuromod.anat.mri...]
get(error): sub-01/ses-002/anat/sub-01_ses-002_bp-cspine_T2w.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(ok): sub-01/ses-002/anat/sub-01_ses-002_bp-cspine_flip-1_mt-off_MTS.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-002/anat/sub-01_ses-002_bp-cspine_flip-1_mt-on_MTS.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-002/anat/sub-01_ses-002_bp-cspine_flip-2_mt-off_MTS.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-002/anat/sub-01_ses-002_flip-1_mt-off_MTS.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-002/anat/sub-01_ses-002_flip-1_mt-on_MTS.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-002/anat/sub-01_ses-002_flip-2_mt-off_MTS.nii.gz (file) [from cneuromod.anat.mri...]
get(error): sub-01/ses-002/anat/sub-01_ses-002_inv-1_MP2RAGE.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(error): sub-01/ses-002/anat/sub-01_ses-002_inv-2_MP2RAGE.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(error): sub-01/ses-002/anat/sub-01_ses-002_rec-dis2d_UNIT1.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(ok): sub-01/ses-002/anat/sub-01_ses-002_rec-dis2d_flip-1_mt-off_MTS.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-002/anat/sub-01_ses-002_rec-dis2d_flip-1_mt-on_MTS.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-002/anat/sub-01_ses-002_rec-dis2d_flip-2_mt-off_MTS.nii.gz (file) [from cneuromod.anat.mri...]
get(error): sub-01/ses-002/anat/sub-01_ses-002_rec-dis2d_inv-1_MP2RAGE.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(error): sub-01/ses-002/anat/sub-01_ses-002_rec-dis2d_inv-2_MP2RAGE.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(ok): sub-01/ses-002/anat/sub-01_ses-002_rec-norm_T1w.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-002/anat/sub-01_ses-002_rec-norm_T2w.nii.gz (file) [from cneuromod.anat.mri...]
get(error): sub-01/ses-002/anat/sub-01_ses-002_rec-norm_mod-T1w_defacemask.nii.gz (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(ok): sub-01/ses-002/dwi/sub-01_ses-002_dir-AP_bp-cspine_dwi.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-002/dwi/sub-01_ses-002_dir-AP_dwi.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-002/dwi/sub-01_ses-002_dir-AP_sbref.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-002/dwi/sub-01_ses-002_dir-PA_dwi.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-002/dwi/sub-01_ses-002_dir-PA_sbref.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-002/fmap/sub-01_ses-002_acq-anat_TB1TFL.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-002/fmap/sub-01_ses-002_acq-famp_TB1TFL.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-002/fmap/sub-01_ses-002_acq-sbref_dir-AP_run-01_epi.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-002/fmap/sub-01_ses-002_acq-sbref_dir-PA_run-01_epi.nii.gz (file) [from cneuromod.anat.mri...]
get(error): sub-01/ses-002/sub-01_ses-002_scans.tsv (file) [not available; (Note that these git remotes have annex-ignore set: origin)]
get(ok): sub-01/ses-002/swi/sub-01_ses-002_minIP.nii.gz (file) [from cneuromod.anat.mri...]
get(ok): sub-01/ses-002/swi/sub-01_ses-002_swi.nii.gz (file) [from cneuromod.anat.mri...]
get(impossible): sub-01/ses-002 (directory) [could not get some content in /Users/julien/temp/neuromod/anat/sub-01/ses-002 ['/Users/julien/temp/neuromod/anat/sub-01/ses-002/anat/sub-01_ses-002_UNIT1.nii.gz', '/Users/julien/temp/neuromod/anat/sub-01/ses-002/anat/sub-01_ses-002_acq-axial_bp-cspine_localizer.nii.gz', '/Users/julien/temp/neuromod/anat/sub-01/ses-002/anat/sub-01_ses-002_acq-axial_localizer.nii.gz', '/Users/julien/temp/neuromod/anat/sub-01/ses-002/anat/sub-01_ses-002_acq-axial_rec-dis2d_localizer.nii.gz', '/Users/julien/temp/neuromod/anat/sub-01/ses-002/anat/sub-01_ses-002_acq-coronal_bp-cspine_localizer.nii.gz', '/Users/julien/temp/neuromod/anat/sub-01/ses-002/anat/sub-01_ses-002_acq-coronal_localizer.nii.gz', '/Users/julien/temp/neuromod/anat/sub-01/ses-002/anat/sub-01_ses-002_acq-coronal_rec-dis2d_localizer.nii.gz', '/Users/julien/temp/neuromod/anat/sub-01/ses-002/anat/sub-01_ses-002_acq-sagittal_bp-cspine_localizer.nii.gz', '/Users/julien/temp/neuromod/anat/sub-01/ses-002/anat/sub-01_ses-002_acq-sagittal_localizer.nii.gz', '/Users/julien/temp/neuromod/anat/sub-01/ses-002/anat/sub-01_ses-002_acq-sagittal_rec-dis2d_localizer.nii.gz', '/Users/julien/temp/neuromod/anat/sub-01/ses-002/anat/sub-01_ses-002_bp-cspine_T2w.nii.gz', '/Users/julien/temp/neuromod/anat/sub-01/ses-002/anat/sub-01_ses-002_inv-1_MP2RAGE.nii.gz', '/Users/julien/temp/neuromod/anat/sub-01/ses-002/anat/sub-01_ses-002_inv-2_MP2RAGE.nii.gz', '/Users/julien/temp/neuromod/anat/sub-01/ses-002/anat/sub-01_ses-002_rec-dis2d_UNIT1.nii.gz', '/Users/julien/temp/neuromod/anat/sub-01/ses-002/anat/sub-01_ses-002_rec-dis2d_inv-1_MP2RAGE.nii.gz', '/Users/julien/temp/neuromod/anat/sub-01/ses-002/anat/sub-01_ses-002_rec-dis2d_inv-2_MP2RAGE.nii.gz', '/Users/julien/temp/neuromod/anat/sub-01/ses-002/anat/sub-01_ses-002_rec-norm_mod-T1w_defacemask.nii.gz', '/Users/julien/temp/neuromod/anat/sub-01/ses-002/sub-01_ses-002_scans.tsv']]
action summary:
get (error: 18, impossible: 1, ok: 25)from cneuromod_docs.
jcohenadad
commented on August 16, 2024
Also: these JSON sidecar files are missing:
sub-01_ses-002_bp-cspine_flip-1_mt-off_MTS.json
sub-01_ses-002_bp-cspine_flip-1_mt-on_MTS.json
sub-01_ses-002_bp-cspine_flip-2_mt-off_MTS.json
(needed for computing MTsat and T1)
from cneuromod_docs.
jcohenadad
commented on August 16, 2024
fresh download (done on 2022-11-08), still getting some errors-- from user facing standpoint this is confusing. See: #83 (comment)
from cneuromod_docs.
jcohenadad
commented on August 16, 2024
tagging @bpinsard
from cneuromod_docs.
bpinsard
commented on August 16, 2024
Should be fixed, now verified by CI/CD testing.
from cneuromod_docs.
Related Issues (20)
- add motion and snr stats
- switch to jupyter book?
- mention informed consent in subjects' description HOT 1
- add dataset contributors for acknowledgements HOT 1
- specify the number of anatomical images in the preprocessing HOT 1
- check if the versions of fmriprep are still correct
- check the number of BOLD runs in the preprocessing docs
- specify we had to adapt the boilerplate information for the preprocessing docs
- add copyright waiver on the documentation, similar to fmriprep
- Push pdf release of the docs on zenodo HOT 1
- create 2022-alpha tag on the doc
- create a summary paragraph of the preprocessing
- improve description of fMRI sequences HOT 2
- create 2022 tag on the docs
- videogame controller link
- Describe the physio signals processing workflow
- hcptrt: additional info
- specify that subjects completed shinobi_training before entering shinobi.
- create a publication gallery
- add stats on dataset size
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cneuromod_docs.