paquiteau / pygpunufft Goto Github PK

gpuNUFFT for python

License: Other

Python 4.92% CMake 1.92% C++ 64.72% Cuda 28.44%

pygpunufft's Introduction

gpuNUFFT - GPU Regridding of arbitrary 3-D/2-D MRI data

This is an hard fork of https://github.com/andyschwarzl/gpuNUFFT , which focuses on providing extensive python bindings and more functionality for non cartesian MRI reconstruction.

Original Software by

Andreas Schwarzl - andy.schwarzl[at]gmail.com
Florian Knoll - florian.knoll[at]nyumc.org

Forked and tailored to python by

Chaythia GR @chaithyagr
Pierre-Antoine Comby @paquiteau

INFO:

GPU 3D/2D regridding library.

REQUIREMENTS:

CUDA capable graphics card and working installation of CUDA Toolkit
pybind11 (downloaded by cmake build)
CMake 3.15 or higher
Google test f(downloaded by cmake build if needed)

CMAKE Options:

WITH_DEBUG : DEFAULT OFF, enables Command-Line DEBUG output
GEN_TESTS : DEFAULT OFF, generate Unit tests

LINUX, using gcc:

build project via cmake, starting from project root directory:

> cd CUDA
> mkdir -p build
> cd build
> cmake ..
> make

Note: This version of gpuNUFFT was tested with CUDA 11.0

Doc:

To generate the source code documentation run

> make doc

in the build directory.

Otherwise, you can go to https://paquiteau.github.io/pygpuNUFFT/docs/ to access the up-to-date documentation of master branch.

Note: Requires doxygen to be installed.

Supporting material

The original written documentation and presentations can be found here.

Acknowlegdment

The authors would like to deeply thank the original authors of gpuNUFFT for their hard work and initial development.

pygpunufft's People

Contributors

Watchers

pygpunufft's Issues

Density Compensation not Working for CUDA 11.2

Density compensation test returns NaN when running with cuda 11.2 and above.

This is maybe due to change in behaviour of cuFFT planning setup and destroy.

Add Documentation of gpuNUFFT algorithms

The source code of the gpuNUFFT is well documented, however the overall process is described elsewhere (for instance in andy Scharwzl bachelor thesis).
This information should be available more closely to the doc (ie in this repo) and have references to the actual implementation (and ideally vice-versa), to ensure a more durable and deeper understanding of the different algorithms in place.

Similarly, The overall process and toolchain used could be documented.

This non-code documentation could be place in the doc/ folder,written in markdown, and publish with doxygen (.md file support has been added in Doxygen 1.8 )

integration with pysap-mri

This repository focuses on python bindings. pysap-mri provides already a wrapper class of the original gpuNUFFT package here. I think this wrapper could be part of the python package provide by pygpuNUFFT.

Moreover, more class could be develop for other usages later on (fMRI reconstruction for instance, density compensation estimation, smaps estimation, etc ...)

Batch gpuNUFFT

-_This is a proposal of implementation for Batch operation for gpuNUFFT, by using multi threading and possibly mutliple gpus.
The multithreading occurs on the "frame" dimension (for instance time frame of fMRI acquisition, but also batches in case Neural Networks).

Kspace locations, Kspace data, density compensation, and Smaps are frame dependent.
Two new parameters are at play and provided by the user

the number of frame n_frame
the number of concurrent operators that a single gpu can run. batch_size

The implementation goes as follows:

a vector of workers (gpuNUFFTOperator instances) is initialized with their respective data. The density compensation is computed if not provided.
each worker is associated a thread (available in STL), and a cuda device number.
then on call of forward or adjoint operator, a pool of batch_size workers are launched.
the result data is written to the same array, at the right place (avoiding race conditions).

Run C++ tests on gpu

The github CI does not have gpu included, so running the C++ test suite (which calls cuda functions) is not possible.

External providers offer such services: google cloud , cirun.io ...

Small Memory leak

When Running a lot of gpuNUFFT operations, one can see that the allocated gpu memory is not entirely freed, and in the end leads to a crash of the process.

In 2d at 32x192x192 resolution after running 800 operators, it in the order of a couple MB, so it has not been critical in our current applications, but it sure will when one wants to scales their program, or have more data to process.

Paralellization over coils

Currently the gpuNUFFT operation are done sequentially over the coil dimension. By using stream concurrency in cuda, its possible to get a maximal throughput of the GPU.
See the attach image for the propose implementation (in the case of data consistency)