Code Monkey home page Code Monkey logo

promptmr's Introduction

PromptMR

This repository contains the pytorch implementation of PromptMR, an unrolled model for multi-coil MRI reconstruction. See our paper Fill the K-Space and Refine the Image: Prompting for Dynamic and Multi-Contrast MRI Reconstruction for more details.

paper PWC

Updates

Method

Overview of PromptMR: an all-in-one unrolled model for MRI reconstruction. Adjacent inputs, depicted in image domain for visual clarity, provide neighboring k-space information for reconstruction. To accommodate different input varieties, the input-type adaptive visual prompt is integrated into each cascade of the unrolled architecture to guide the reconstruction process.

Overview of the PromptUnet: the denoiser in each cascade of PromptMR. The PromptBlocks can generate adaptively learned prompts across multiple levels, which integrate with decoder features in the UpBlocks to allow rich hierachical context learning.

Installation and Data Preparation

See INSTALL.md for installation instructions and data preparation required to run this codebase.

Training/Inference Codes & Pretrained models

CMRxRecon dataset

FastMRI multi-coil knee dataset

Citation

If you found this repository useful to you, please consider giving a star ⭐️ and citing our paper:

@article{xin2023fill,
  title={Fill the K-Space and Refine the Image: Prompting for Dynamic and Multi-Contrast MRI Reconstruction},
  author={Xin, Bingyu and Ye, Meng and Axel, Leon and Metaxas, Dimitris N},
  journal={arXiv preprint arXiv:2309.13839},
  year={2023}
}

Acknowledgements

E2E-VarNet, HUMUS-Net, PromptIR, CMRxRecon

statistics

visitors

promptmr's People

Contributors

hellopipu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

promptmr's Issues

About Acceleration Rates

Hello,

In the context of both cardiac and knee challenges, there are multiple acceleration rates available. For cardiac scans, there are 4, 8, and 10-fold accelerated scans, and for knee scans, there are 4 and 8-fold accelerated scans. I'm seeking clarification on the training and testing procedure for the network:

Was the network trained exclusively with 4-fold accelerated scans and then tested with 4, 8, and 10-fold accelerated scans?

Or, was the network trained with 4-fold accelerated scans and tested only with 4-fold accelerated test scans?

Additionally, I'd like to know if the same approach extends to other acceleration rates.

Thank you,

About code on calgary-campinas

Thanks for your amazing work, which has inspired me a lot!
I am currently trying to apply PromptMR to the calgary-campinas dataset and examine its effects. However, I am facing some difficulties in processing the data and adapting it to the format required by the model. If possible, could you please share the code for training PromptMR on this dataset?
I appreciate it very much !

hello!I have encountered this problem

When I run the pretrained file, an error occurs:
Traceback (most recent call last):
File "run_pretrained_promptmr_fastmri_knee_inference.py", line 182, in
torch.device(args.device),
File "run_pretrained_promptmr_fastmri_knee_inference.py", line 121, in run_inference
model.load_state_dict(torch.load(state_dict_file))
File "/home/zhengkaiyi/anaconda3/envs/zky/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1672, in load_state_dict
self.class.name, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for PromptMR:
Is it because I ran the files in the wrong order, or is there another reason? Could you help me solve it?

Training Image sizes

Hello,

What spatial dimensions are you using for training on the CMRxRecon dataset? I didn't find any cropping or padding in the data loaders for it.

Is it purely through asymmetric padding in the Unit Layers for the full-resolution scans? [144x512 or 116x382 for the T1/T2 task]

Support for Non-Cartesian Trajectories in Prompt

Hello,

I am interested in using non-Cartesian trajectories (radial, spiral) in imaging and would like to know if PromptMR can support these types of trajectories. This feature could add significant value to our current workflows.

Best,

GPU memory for training

Hi,
Thanks for sharing the code of this interesting work.

I am trying to run the training on the fastMRI dataset and I got CUDA out of memory issue even with batch size=1.
My GPU is NVIDIA A5000, which has 24G memory.

Could you please tell me how much GPU memory is required to train with batchsize=1?

BTW, I noticed that the memory is gradually increasing for each iteration (batch).
Is that normal? Maybe this is somehow related to the code itself and I didn't notice.

Many thanks! looking forward to your reply.

About Checkpoint

Hello,

Is it possible to share 4x acceleration knee checkpoint ? I want to compare it with ensemble methods and prospective other tasks.

Thanks,

Validation set results

Hello,

Thank you very much for your effort and congratulations for this amazing achievement.

I have two questions if you allow me.
1) Regrading the validation set, the challenge submission leaderboard is closed now. May I ask if is there a way to reproduce
the results you reported in the paper (I did the training already)?
2) May I know the system you used for the training please?

Thank you very much for your time.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.