arnab39 / fewshot_gan-unet3d Goto Github PK

Tensorflow implementation of our paper: Few-shot 3D Multi-modal Medical Image Segmentation using Generative Adversarial Learning

License: MIT License

Python 99.98% Shell 0.02%

few-shot-learning brain-mri semi-supervised-learning unet-3d unet-tensorflow gans semi-supervised-gan multi-modal-imaging medical-image-analysis iseg-challenge

fewshot_gan-unet3d's Introduction

Few-shot 3D Medical Image Segmentation using Generative Adversarial Learning

This repository contains the tensorflow and pytorch implementation of the model we proposed in our paper of the same name: Few-shot 3D Medical Image Segmentation using Generative Adversarial Learning

The code is available in both tensorflow and pytorch. To run the project kindly refer to the individual readme file.

Dataset

iSEG 2017 dataset was chosen to substantiate our proposed method. It contains the 3D multi-modal brain MRI data of 10 labeled training subjects and 13 unlabeled testing subjects. We split the 10 labeled training data into training, validation and testing images for both the models.(Eg- 2,1 and 7) Rest of the 13 unlabeled testing images are only used for training the GAN based model.

MRBrainS13 dataset was also used to test the robustness of our proposed model. It contains 3 modalities (T1-weighted, T1-weighted inversion recovery and FLAIR). Original Dataset consists of 5 labeled training and 6 unlabeled testing subjects. We split the 5 labeled training data into 1,1,3 for training, validation and testing. The 6 testing subjects are used as unlabeled data for the GAN based models.

Proposed model architecture

The following shows the model architecture of the proposed model. (Read our paper for further details)

Some results from our paper

Visual comparison of the segmentation by each model, for two test subjects of the iSEG-2017 dataset, when training with different numbers of labeled examples.

Segmentation of Subject 10 of the iSEG-2017 dataset predicted by different GAN-based models, when trained with 2 labeled images. The red box highlights a region in the ground truth where all these models give noticeable differences.

* More such results can be found in the paper.

Contact

You can mail us at: [email protected], [email protected] If you use this code for your research, please consider citing the original paper:

Arnab Kumar Mondal, Jose Dolz, Christian Desrosiers. Few-shot 3D Multi-modal Medical Image Segmentation using Generative Adversarial Learning.

fewshot_gan-unet3d's People

Contributors

Stargazers

Watchers

Forkers

harshacse sandeep-sm hsouporto sohailkhanmarwat decoderkurt huangpu1 mdsharique xiaochengcike cathy-kim ieee820 abnerxzhe gaimjkp rsip4sh zwq1230 victor8733 gchoi monchhichizzq liujie1977 lewisget stellali-0501 dl-alva alanmorninglight ihsan149 equifai essie-chiang const-yield kristin-zlchen wentaohub zlinzju baobunuo psicobloc nanyomy zwlshine xhongz licj1 dlyzhou annecurie liminenjoy bubblyyi xub69961415 panxipeng kavitashinde24 iact-medical-image-processing ifv lxmwust hkthirano ximenadai suyouooooo crie12138 qianliu1219 marcelo-bm-santos dhinkris yeqiuyi saqibmamoon guohuigao sergivalverde aigookdo phillipchoi007 walehn bmquynhlinh tengtengzhong superrenh pjm358 blacksun-v tigerdavid ehdgns21 hurongjie ritaxuu yongkangjun csie-lab-aug2021 markyang19 hamyadkiani gabriellaaileen hadinursalim guangwensi muyangly rian-day junha0zhang donglongzi lustzero shizoda

fewshot_gan-unet3d's Issues

N4BiasField

OSError: No command "N4BiasFieldCorrection" found on host tx-eva-stianci. Please check that the corresponding package is installed.

Could anyone share dataset? I fail to register and download the dataset. Thanks.

Replacing U-Net with Isensee Model

Did you try replacing U-Net with a new upgraded U-Net version called Isensee Model+3DGAN? How do you think the results may vary?

ImportError: cannot import name 'extract_patches' from 'sklearn.feature_extraction.image' (/home/nakayama/.local/lib/python3.9/site-packages/sklearn/feature_extraction/image.py)

$python main.py configs/unet.json
The error was happended.
What should you do?

Why do I get the same loss every validation ?Doesn't the model change?

I get the loss of the validation:17.218517 for the first time ,but afterwards it doesn't change even a little,I completely follow the instructions of the program . Is it the problem that I ignore some details or the program get some little problem with the model ?

I appreciate this is a great work, I'd like to konw what data used to train the model? 1 labeled subject and another one unlabeled subject, is it right? and the unlabeled subject also used to train the GAN?

Num_Spatial_Dims argument missing

Error message:
\FewShot_GAN-Unet3D-master\pytorch\graphs\models\custom_functions\weight_norm.py",
line 165, in forward
output_padding = self._output_padding(input, output_size, self.stride, self.padding,
self.kernel_size)
TypeError: _output_padding() missing 1 required positional argument: 'num_spatial_dims'

It states that the num_spatial_dims argument is missing from line 165 in the weight_norm python file (within the 'forward' definition). What should this value for nums_spatial_dims be? Thank you!

Use which command to test the proposed model? I thibk there are some fauts in the text. It should be $ python main.py --testing not $ python main_unet.py --testing, is it right?

ValueError: could not broadcast input array from shape (144,192,256,1) into shape (144,192,256)

Hi, I have covert .img to .nii, but when i run python main.py(bad gan), the error is
File "E:/slycode\FewShot_GAN-Unet3D-master\preprocess\preprocess.py", line 305, in init
patch_shape,number_images_training)
File "E:/slycode\FewShot_GAN-Unet3D-master\preprocess\preprocess.py", line 162, in preprocess_dynamic_lab
T1_vols[(case_idx-c-1), :, :, :] = read_vol(case_idx, 'T1', dir)
ValueError: could not broadcast input array from shape (144,192,256,1) into shape (144,192,256)
should i change the (144,192,256) in the preprocess.py to (144,192,256,1)???

CUDA error: device-side assert triggered, The loss here is a tensor, and no value is returned.

CUDA error: device-side assert triggered

The iseg2017 data is used, and the data format is hdr. The loss here is a tensor, and no value is returned. How to solve this problem?
run FewShot_GAN-Unet3D-master/pytorch/agents/supervised_baseline.py

self.criterion =nn.CrossEntropyLoss(class_weights)
loss = self.criterion(output_logits, labels)
loss.backward()
self.optimizer.step()

running python main.py --training getting error.

I am getting error . What can be the problem here.

`Training
1
../data/iSEG_preprocessed/Training/T1/subject-1-T1.nii
../data/iSEG_preprocessed/Training/T2/subject-1-T2.nii
../data/iSEG_preprocessed/Training/label/subject-1-label.nii
Extracting Patches from Image 1 ....
Total Extracted Labelled Patches Shape: (2605, 32, 32, 32, 2) (2605, 32, 32, 32)
../data/iSEG_preprocessed/Testing/T1/subject-11-T1.nii
../data/iSEG_preprocessed/Testing/T2/subject-11-T2.nii
../data/iSEG_preprocessed/Training/label/subject-1-label.nii
Processing the Image 11 ....
Traceback (most recent call last):
File "main.py", line 81, in
tf.app.run()
File "/home/hmahmood/anaconda3/envs/pi352/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 126, in run
_sys.exit(main(argv))
File "main.py", line 73, in main
network.train()
File "/home/hmahmood/FewShot_GAN-Unet3D/proposed_model/model.py", line 223, in train
data_directory=F.data_directory)
File "../preprocess/preprocess.py", line 311, in init
patch_shape, number_unlab_images_training)

File "../preprocess/preprocess.py", line 246, in preprocess_dynamic_unlab
x=get_patches_unlab(T1_vols, T2_vols, extraction_step, patch_shape,dir)
File "../preprocess/preprocess.py", line 205, in get_patches_unlab
label_patches = extract_patches(label_ref, patch_shape, extraction_step)
File "../preprocess/preprocess.py", line 81, in extract_patches
dstride_d:(dstride_d)+patch_d]
ValueError: could not broadcast input array from shape (32,32,32,1) into shape (32,32,32)`

def_init_(dump)

File "C:\Users\virkt\Anaconda3\envs\unet\lib\multiprocessing\reduction.py", line 244
def init(dump)
^
SyntaxError: invalid syntax

运行main_unet.py出错

我想问一下那个model_unet.py函数中259行的那个recompose3D_overlap函数是怎么定义的，源程序好像没有定义，请教大神，我的邮箱是[email protected]，QQ号是2919028238

How did you find class weights?

Dear Arnab,

It's not clear for me how did you choose the class weights? Did you estimate weights similar to compute_class_weight only from the training data? Did you fine-tune the weights manually?

Background- 0.33, CSF- 1.5, GM- 0.83, WM- 1.33: https://github.com/arnab39/FewShot_GAN-Unet3D/blob/master/unet3D/model_unet.py#L152.

btw Thanks a lot for sharing your code and paper. Sadly, there was nothing said about class weights in your paper.

Best regards,
Stas

Ask for help！Thanks a lot！

when I 'run main.py --testing' have some errors FailedPreconditionError (see above for traceback): Attempting to use uninitialized value D/d_up3_deconv/g [[node D/d_up3_deconv/g/read (defined at ../lib/operations.py:87) = Identity[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"](D/d_up3_deconv/g)]] [[{{node ArgMax/_3}} = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_491_ArgMax", tensor_type=DT_INT64, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

Incorporate into related toolkits

Your approach is interesting. Please consider having it incorporated into the following toolkits if you haven't already:

https://github.com/NifTK/NiftyNet
https://github.com/perone/medicaltorch
https://github.com/pfjaeger/medicaldetectiontoolkit

Doing this can dramatically increase the audience of your approach. Thanks.

Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR

I got the following error when running python main_unset.py --training

2019-09-12 21:17:30.594167: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 AVX512F FMA
2019-09-12 21:17:31.140393: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1405] Found device 0 with properties: 
name: GeForce RTX 2080 Ti major: 7 minor: 5 memoryClockRate(GHz): 1.545
pciBusID: 0000:1a:00.0
totalMemory: 10.73GiB freeMemory: 10.57GiB
2019-09-12 21:17:31.324298: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1405] Found device 1 with properties: 
name: GeForce RTX 2080 Ti major: 7 minor: 5 memoryClockRate(GHz): 1.545
pciBusID: 0000:1b:00.0
totalMemory: 10.73GiB freeMemory: 10.57GiB
2019-09-12 21:17:31.508353: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1405] Found device 2 with properties: 
name: GeForce RTX 2080 Ti major: 7 minor: 5 memoryClockRate(GHz): 1.545
pciBusID: 0000:3d:00.0
totalMemory: 10.73GiB freeMemory: 10.57GiB
2019-09-12 21:17:31.691047: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1405] Found device 3 with properties: 
name: GeForce RTX 2080 Ti major: 7 minor: 5 memoryClockRate(GHz): 1.545
pciBusID: 0000:3e:00.0
totalMemory: 10.73GiB freeMemory: 10.57GiB
2019-09-12 21:17:31.691543: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1484] Adding visible gpu devices: 0, 1, 2, 3
2019-09-12 21:17:33.385536: I tensorflow/core/common_runtime/gpu/gpu_device.cc:965] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-09-12 21:17:33.385595: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971]      0 1 2 3 
2019-09-12 21:17:33.385603: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] 0:   N N N N 
2019-09-12 21:17:33.385608: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] 1:   N N N N 
2019-09-12 21:17:33.385613: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] 2:   N N N N 
2019-09-12 21:17:33.385618: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] 3:   N N N N 
2019-09-12 21:17:33.386348: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1097] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10439 MB memory) -> physical GPU (device: 0, name: GeForce RTX 2080 Ti, pci bus id: 0000:1a:00.0, compute capability: 7.5)
2019-09-12 21:17:33.589232: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1097] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:1 with 10439 MB memory) -> physical GPU (device: 1, name: GeForce RTX 2080 Ti, pci bus id: 0000:1b:00.0, compute capability: 7.5)
2019-09-12 21:17:33.799606: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1097] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:2 with 10439 MB memory) -> physical GPU (device: 2, name: GeForce RTX 2080 Ti, pci bus id: 0000:3d:00.0, compute capability: 7.5)
2019-09-12 21:17:33.995857: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1097] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:3 with 10439 MB memory) -> physical GPU (device: 3, name: GeForce RTX 2080 Ti, pci bus id: 0000:3e:00.0, compute capability: 7.5)
2019-09-12 21:18:10.689782: E tensorflow/stream_executor/cuda/cuda_dnn.cc:352] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR
Fatal Python error: Segmentation fault

Thread 0x00007f1e85266740 (most recent call first):
  File "/home/skyuuka/anaconda2/envs/FewShot_GAN-Unet3D/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1350 in _call_tf_sessionrun
  File "/home/skyuuka/anaconda2/envs/FewShot_GAN-Unet3D/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1263 in _run_fn
  File "/home/skyuuka/anaconda2/envs/FewShot_GAN-Unet3D/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1278 in _do_call
  File "/home/skyuuka/anaconda2/envs/FewShot_GAN-Unet3D/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1272 in _do_run
  File "/home/skyuuka/anaconda2/envs/FewShot_GAN-Unet3D/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1100 in _run
  File "/home/skyuuka/anaconda2/envs/FewShot_GAN-Unet3D/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 877 in run
  File "/home/skyuuka/anaconda2/envs/FewShot_GAN-Unet3D/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 4986 in _run_using_default_session
  File "/home/skyuuka/anaconda2/envs/FewShot_GAN-Unet3D/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2241 in run
  File "/home/skyuuka/workspace/FewShot_GAN-Unet3D/unet3D/model_unet.py", line 213 in train
  File "main_unet.py", line 62 in main
  File "/home/skyuuka/anaconda2/envs/FewShot_GAN-Unet3D/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 125 in run
  File "main_unet.py", line 69 in <module>

using my own dataset

how to train on my own dataset?