Light

marksgraham / dw-possum Goto Github PK

View Code? Open in Web Editor NEW

18.0 7.0 7.0 94 KB

Simulate realistic diffusion-weighted MR data

License: MIT License

Python 96.67% Shell 3.33%

possum diffusion-mri simulation

dw-possum's Introduction

This code enables the simulation of DW-MR data using the POSSUM simulator.

Dependencies

FSL 5.0.10 - DW-POSSUM has only been tested with this version
Python with dipy, nibabel, numpy and scipy

Setup

Clone this repository
Run bash getFiles.sh to download the imaging data

Quick-start tutorial

DW-POSSUM is run in three stages. First all the files required for the simulation are generated (generateFileStructure.py). Then the simulations are run (runPossum.py). Finally everything is tidied up (postProcess.py). The following commands will simulate the acquistion of a single slice for the first three entries in the bval/bvec file supplied (two b=0 and one b=1000 volumes).

Generate required files

./generateFileStructure.py Files/POSSUMdirectories/possumSimdirOneSlice/ Test/ Files/Bvalsbvecs/bvalsfmrib Files/Bvalsbvecs/bvecsfmrib --num_images 3

Run POSSUM

./runPossum.py Test 3

Tidy up

./postProcess.py Test 3 --noise_levels 0 0.0081 0.0165

Your simulations will be stored as 4D nifti files in Test/Results/.

Documentation for each command can be found using the -h flag.

Computational requirements

generateFileStructure.py is fairly memory hungry - it may require up to 12GB RAM.
runPossum.py is where the meat of the simulation happens, and is very CPU intensive. Under the hood, it calls the FSL command possumX. For large simulations (i.e. anything more than a few slices of the brain) I recommend running this on a cluster environment that has been set up for automatic self-submission of FSL jobs (more information here). This will automatically divide each simulation into a number of jobs (the number can be set using the --num_processors flag), submit each to the cluster and handle recombination into the final image volume. On local systems you can easily get POSSUM to take advantage of multiple cores using this - make sure you set the --num_processors flag equal to the number of available cores.

Datasets

Some of the simulated datasets used in the journal papers referenced below can be downloaded here.

References

If you use DW-POSSUM in your research, please cite both the POSSUM and DW-POSSUM papers:

Ivana Drobnjak, David Gavaghan, Endre Süli, Joe Pitt-Francis and Mark Jenkinson. Development of a functional magnetic resonance imaging simulator for modeling realistic rigid-body motion artifacts, MRM 56 364–380, 2006.
Mark S. Graham, Ivana Drobnjak and Hui Zhang. Realistic simulation of artefacts in diffusion MRI for validating post-processing correction techniques, NeuroImage 125, 1079-1094, 2016.

We have extended POSSUM to simulate spin-echo sequences, enabling (in combination with this codebase) the simulation of DW-MR data with susceptibility artefacts. The new POSSUM release is forthcoming, though the code can already be found here and some simulated DW-MR datasets with susceptibility artefacts have been made available here. The relevant reference for this work is:

Mark S. Graham, Ivana Drobnjak, Mark Jenkinson and Hui Zhang. Quantitative assessment of the susceptibility artefact and its interaction with motion in diffusion MRI, PLoS ONE 12(10): e0185647, 2017.

dw-possum's People

Contributors

Stargazers

Watchers

Forkers

snowbhr06 jhlegarreta prabhjot21 rmukh slieped ywangc ricardozitseng

dw-possum's Issues

spikes in epi pulse sequence

Hi Mark, when I visualize the EPI pulse sequence used for DWI simulation included in your Files, I saw there're spikes at the beginning and end of all three gradients (Gx, Gy and Gz), I wonder how should I interpret those spikes? For the front ones I guess they are to move to the initial k-space starting point? Which is understandable. But how about the end ones?

From some textbooks it seems there should not be such spikes, so how would these affect simulation results?

Error running step 3 - postprocess

Hi there, I can run the first two steps without problems, however when it comes to the last step it gives the following error message and I didn't see any generated files under directory /Test/Results
➜ DW-POSSUM git:(master) ✗ ./postProcess.py Test 3 --noise_levels 0 0.0081 0.0165
[0.0, 0.0081, 0.0165]
nx,ny,nz 72 86 1
dx,dy,dz 2.5 2.5 2.5
Reshaping the signal...

error: Mat::operator(): index out of bounds

terminate called after throwing an instance of 'std::logic_error'
what(): Mat::operator(): index out of bounds
mv: cannot stat '/hd2/EE591/project/DW-POSSUM/Test/Direction0/imageNoise_abs.nii.gz': No such file or directory
Image Exception : #63 :: No image files match: /hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0_image0
terminate called after throwing an instance of 'armawrap::AWException'
what(): No image files match: /hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0_image0
nx,ny,nz 72 86 1
dx,dy,dz 2.5 2.5 2.5
Reshaping the signal...

error: Mat::operator(): index out of bounds

terminate called after throwing an instance of 'std::logic_error'
what(): Mat::operator(): index out of bounds
mv: cannot stat '/hd2/EE591/project/DW-POSSUM/Test/Direction0/imageNoise_abs.nii.gz': No such file or directory
Image Exception : #63 :: No image files match: /hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0081_image0
terminate called after throwing an instance of 'armawrap::AWException'
what(): No image files match: /hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0081_image0
nx,ny,nz 72 86 1
dx,dy,dz 2.5 2.5 2.5
Reshaping the signal...

error: Mat::operator(): index out of bounds

terminate called after throwing an instance of 'std::logic_error'
what(): Mat::operator(): index out of bounds
mv: cannot stat '/hd2/EE591/project/DW-POSSUM/Test/Direction0/imageNoise_abs.nii.gz': No such file or directory
Image Exception : #63 :: No image files match: /hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0165_image0
terminate called after throwing an instance of 'armawrap::AWException'
what(): No image files match: /hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0165_image0
nx,ny,nz 72 86 1
dx,dy,dz 2.5 2.5 2.5
Reshaping the signal...

error: Mat::operator(): index out of bounds

terminate called after throwing an instance of 'std::logic_error'
what(): Mat::operator(): index out of bounds
mv: cannot stat '/hd2/EE591/project/DW-POSSUM/Test/Direction1/imageNoise_abs.nii.gz': No such file or directory
Image Exception : #63 :: No image files match: /hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0_image1
terminate called after throwing an instance of 'armawrap::AWException'
what(): No image files match: /hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0_image1
nx,ny,nz 72 86 1
dx,dy,dz 2.5 2.5 2.5
Reshaping the signal...

error: Mat::operator(): index out of bounds

terminate called after throwing an instance of 'std::logic_error'
what(): Mat::operator(): index out of bounds
mv: cannot stat '/hd2/EE591/project/DW-POSSUM/Test/Direction1/imageNoise_abs.nii.gz': No such file or directory
Image Exception : #63 :: No image files match: /hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0081_image1
terminate called after throwing an instance of 'armawrap::AWException'
what(): No image files match: /hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0081_image1
nx,ny,nz 72 86 1
dx,dy,dz 2.5 2.5 2.5
Reshaping the signal...

error: Mat::operator(): index out of bounds

terminate called after throwing an instance of 'std::logic_error'
what(): Mat::operator(): index out of bounds
mv: cannot stat '/hd2/EE591/project/DW-POSSUM/Test/Direction1/imageNoise_abs.nii.gz': No such file or directory
Image Exception : #63 :: No image files match: /hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0165_image1
terminate called after throwing an instance of 'armawrap::AWException'
what(): No image files match: /hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0165_image1
nx,ny,nz 72 86 1
dx,dy,dz 2.5 2.5 2.5
Reshaping the signal...

error: Mat::operator(): index out of bounds

terminate called after throwing an instance of 'std::logic_error'
what(): Mat::operator(): index out of bounds
mv: cannot stat '/hd2/EE591/project/DW-POSSUM/Test/Direction2/imageNoise_abs.nii.gz': No such file or directory
Image Exception : #63 :: No image files match: /hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0_image2
terminate called after throwing an instance of 'armawrap::AWException'
what(): No image files match: /hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0_image2
nx,ny,nz 72 86 1
dx,dy,dz 2.5 2.5 2.5
Reshaping the signal...

error: Mat::operator(): index out of bounds

terminate called after throwing an instance of 'std::logic_error'
what(): Mat::operator(): index out of bounds
mv: cannot stat '/hd2/EE591/project/DW-POSSUM/Test/Direction2/imageNoise_abs.nii.gz': No such file or directory
Image Exception : #63 :: No image files match: /hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0081_image2
terminate called after throwing an instance of 'armawrap::AWException'
what(): No image files match: /hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0081_image2
nx,ny,nz 72 86 1
dx,dy,dz 2.5 2.5 2.5
Reshaping the signal...

error: Mat::operator(): index out of bounds

terminate called after throwing an instance of 'std::logic_error'
what(): Mat::operator(): index out of bounds
mv: cannot stat '/hd2/EE591/project/DW-POSSUM/Test/Direction2/imageNoise_abs.nii.gz': No such file or directory
Image Exception : #63 :: No image files match: /hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0165_image2
terminate called after throwing an instance of 'armawrap::AWException'
what(): No image files match: /hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0165_image2
Image Exception : #63 :: No image files match: /hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0_image0
terminate called after throwing an instance of 'armawrap::AWException'
what(): No image files match: /hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0_image0
Aborted (core dumped)
rm: cannot remove '/hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0_image0.nii.gz': No such file or directory
rm: cannot remove '/hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0_image1.nii.gz': No such file or directory
rm: cannot remove '/hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0_image2.nii.gz': No such file or directory
Image Exception : #63 :: No image files match: /hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0081_image0
terminate called after throwing an instance of 'armawrap::AWException'
what(): No image files match: /hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0081_image0
Aborted (core dumped)
rm: cannot remove '/hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0081_image0.nii.gz': No such file or directory
rm: cannot remove '/hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0081_image1.nii.gz': No such file or directory
rm: cannot remove '/hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0081_image2.nii.gz': No such file or directory
Image Exception : #63 :: No image files match: /hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0165_image0
terminate called after throwing an instance of 'armawrap::AWException'
what(): No image files match: /hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0165_image0
Aborted (core dumped)
rm: cannot remove '/hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0165_image0.nii.gz': No such file or directory
rm: cannot remove '/hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0165_image1.nii.gz': No such file or directory
rm: cannot remove '/hd2/EE591/project/DW-POSSUM/Test/Results/diff_sigma0.0165_image2.nii.gz': No such file or directory

excuse me , I want to know how to simulate the artefacts MRI data?

excuse me , I want to know how to simulate the artefacts MRI data?

Error when running step 2

/usr/local/fsl/bin/fsl_sub -T 20 -l /data/jht/DW-POSSUM-master/Test/Direction0/logs -M [email protected] -N possum -t /data/jht/DW-POSSUM-master/Test/Direction0/possum.com
Error submitting job - Errors occured when running array task in shell plugin: Task 1 failed executing: /usr/local/fsl/bin/possum -i /data/jht/DW-POSSUM-master/Test/Direction0/brain -x /data/jht/DW-POSSUM-master/Test/Direction0/MRpar -f /data/jht/DW-POSSUM-master/Test/Direction0/slcprof -p /data/jht/DW-POSSUM-master/Test/Direction0/pulse -m /data/jht/DW-POSSUM-master/Test/Direction0/motion --nproc=1 -e /data/jht/DW-POSSUM-master/Test/Direction0/matrix --procid=0 -o /data/jht/DW-POSSUM-master/Test/Direction0/diff_proc/signal_proc_0 (Image Exception : #22 :: Failed to read volume /data/jht/DW-POSSUM-master/Test/Direction0/brain
Error : Error: ROI out of bounds for /data/jht/DW-POSSUM-master/Test/Direction0/brain.nii.gz
terminate called after throwing an instance of 'std::runtime_error'
what(): Failed to read volume /data/jht/DW-POSSUM-master/Test/Direction0/brain
Error : Error: ROI out of bounds for /data/jht/DW-POSSUM-master/Test/Direction0/brain.nii.gz
)

/usr/local/fsl/bin/fsl_sub -T 10 -j -F -l /data/jht/DW-POSSUM-master/Test/Direction0/logs /usr/local/fsl/bin/possumX_postproc.sh /data/jht/DW-POSSUM-master/Test/Direction0 1
usage: fsl_sub [-h] [-a ARCH] [-c COPROCESSOR]
[--coprocessor_multi COPROCESSOR_MULTI]
[--coprocessor_class COPROCESSOR_CLASS]
[--coprocessor_class_strict]
[--coprocessor_toolkit COPROCESSOR_TOOLKIT] [-F] [-j JOBHOLD]
[--not_requeueable] [--array_hold ARRAY_HOLD] [-l LOGDIR]
[-m MAILOPTIONS] [-M MAILTO] [-n] [-N NAME] [-p PRIORITY]
[-q QUEUE] [-r RESOURCE] [--delete_job DELETE_JOB]
[--extra EXTRA] [-R GB] [-s PARALLELENV,THREADS]
[-t ARRAY_TASK] [--array_native ARRAY_NATIVE] [-x NUMBER]
[--keep_jobscript] [--has_coprocessor COPROCESSOR_NAME]
[--has_queues] [--project PROJECT] [-S] [-T MINUTES]
[--show_config] [-v] [-V] [-z file]
...
fsl_sub: error: argument -j/--jobhold: expected one argument

Code isn't python 3 compatible

Code still contains vestiges of 'list formatting'

e.g. numImagesList[0] rather than numImages

Error running step 3 - postprocess #4 (2)

Dear Marks Graham,

I have to reopen this issue since I am trying to understand how to run the FSL-POSSUM.
As reference issue #4
I have the same problem.
Even if I change my environment variable FSLDIR to /fsl5.0
I still can not work through step 3.
Here is my output of step 2:
./runPossum.py Test 3

subjectdir is /qmi_home/anht/Desktop/workspace/BIG_protocol/fslPOSSUM/DWPOSSUM_master/Test/Direction0
Making possum directory structure
Preprocessing stage
/raid1b/STBBapps/fsl5.0/bin/possum_matrix -p /qmi_home/anht/Desktop/workspace/BIG_protocol/fslPOSSUM/DWPOSSUM_master/Test/Direction0/pulse -m /qmi_home/anht/Desktop/workspace/BIG_protocol/fslPOSSUM/DWPOSSUM_master/Test/Direction0/motion --seg=1000 -o /qmi_home/anht/Desktop/workspace/BIG_protocol/fslPOSSUM/DWPOSSUM_master/Test/Direction0/matrix
Processing stage

Now going to step 3:
./postProcess.py Test 3 --noise_levels 0 0.0081 0.0165

qmi_home/anht/anaconda3/lib/python3.6/site-packages/dicom/init.py:53: UserWarning:
This code is using an older version of pydicom, which is no longer
maintained as of Jan 2017. You can access the new pydicom features and API
by installing pydicom from PyPI.
See 'Transitioning to pydicom 1.x' section at pydicom.readthedocs.org
for more information.
warnings.warn(msg)
[0.0, 0.0081, 0.0165]
nx,ny,nz 72 86 1
dx,dy,dz 2.5 2.5 2.5
Reshaping the signal...
error: Mat::operator(): index out of bounds
terminate called after throwing an instance of 'std::logic_error'
what(): Mat::operator(): index out of bounds
mv: cannot stat '/qmi_home/anht/Desktop/workspace/BIG_protocol/fslPOSSUM/DWPOSSUM_master/Test/Direction0/imageNoise_abs.nii.gz': No such file or directory
Image Exception : #63 :: No image files match: /qmi_home/anht/Desktop/workspace/BIG_protocol/fslPOSSUM/DWPOSSUM_master/Test/Results/diff_sigma0.0_image0
terminate called after throwing an instance of 'std::runtime_error'
what(): No image files match: /qmi_home/anht/Desktop/workspace/BIG_protocol/fslPOSSUM/DWPOSSUM_master/Test/Results/diff_sigma0.0_image0

I am currently using python 3.7. Is there any way to work around it.
Thank you.

What is the relationship between noise_levels and SNR in step 3

./postProcess.py Test 3 --noise_levels 0 0.0081 0.0165
For example, what is the SNR corresponding to 0.0081

index and acparam of simulated DWI for input of eddy

Hi, I'm trying to use simulated dwi published in https://www.nitrc.org/projects/diffusionsim/ which is said to be generated by DW-possum for experiments with FSL eddy's outlier detection function. There're two parameters required by eddy, index and acparam specifying information about phase encoding. What should I input to eddy if I use this simulated DWI? Or does it matter what's the content of these two files if I'm only interested in eddy's outlier detection function?

How to run with Spin echo possum

Dear Mark Graham

I am trying to simulate DW data with artifacts from eddy current, motion and susceptibility distortions.

You said that "We have extended POSSUM to simulate spin-echo sequences, enabling (in combination with this codebase) the simulation of DW-MR data with susceptibility artefacts". So, I have extended my installation of FSL with the files provided in the spin echo possum github and it seems to be ok.

My problem is that I don't really see what to change with the commands of DW-POSSUM to simulate also the susceptibility now that I have SE-Possum. Do I need extra files and is there default files to test ?
For example do I need to change the MRpar manually each time ?

It also seems that I have problems to simulate the motion with DW-Possum. The file 'Test/Distortions/Motion/' stays empty.

Thank you in advance
Simulate DW data will be very useful for my Master's thesis :)

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.