angeloucn / min_max_similarity Goto Github PK

A contrastive learning based semi-supervised segmentation network for medical image segmentation

Python 100.00%

contrastive-learning semi-supervised-learning medical-image-segmentation surgical-tools-segmentation medical-video-analysis video-segmentation

min_max_similarity's Introduction

Min_Max_Similarity

A contrastive learning based semi-supervised segmentation network for medical image segmentation This repository contains the implementation of a novel contrastive learning based semi-segmentation networks to segment the surgical tools.

Fig. 1. The architecture of Min-Max Similarity.

🔥 NEWS 🔥 The full paper is available: Min-Max Similarity

🔥 NEWS 🔥 The paper has been accepted by IEEE Transactions on Medical Imaging. The early access is available at Here.

Environment

python==3.6
packages:

conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=11.1 -c pytorch -c conda-forge

conda install opencv-python pillow numpy matplotlib

Clone this repository

git clone https://github.com/AngeLouCN/Min_Max_Similarity

Data Preparation

We use five dataset to test its performance:

File structure

|-- data
|   |-- kvasir
|   |   |-- train
|   |   |   |--image
|   |   |   |--mask
|   |   |-- test
|   |   |   |--image
|   |   |   |--mask
|   |-- EndoVis17
|   |   |-- train
|   |   |   |--image
|   |   |   |--mask
|   |   |-- test
|   |   |   |--image
|   |   |   |--mask
......

You can also test on some other public medical image segmentation dataset with above file architecture

Usage

Training: You can change the hyper-parameters like labeled ratio, leanring rate, and e.g. in train_mms.py, and directly run the code.
Testing: You can change the dataset name in test.py and run the code.

Segmentation Performance

Fig. 2. Visual comparison of our method with state-of-the-art models. Segmentation results are shown for 50% of labeled training data for Kvasir-instrument, EndVis’17, ART-NET and RoboTool, and 2.4% labeled training data for cochlear implant. From left to right are EndoVis’17, Kvasir-instrument, ART-NET, RoboTool, Cochlear implant and region of interest (ROI) of Cochlear implant.

Citation

@article{lou2023min,
  title={Min-Max Similarity: A Contrastive Semi-Supervised Deep Learning Network for Surgical Tools Segmentation},
  author={Lou, Ange and Tawfik, Kareem and Yao, Xing and Liu, Ziteng and Noble, Jack},
  journal={IEEE Transactions on Medical Imaging},
  year={2023},
  publisher={IEEE}
}

Acknowledgement

Our code is based on the Duo-SegNet, we thank their excellent work and repository.

min_max_similarity's People

Contributors

Stargazers

Watchers

Forkers

xyimaging hardy-jin fabrizzio-gzb zengjixiang2eziw itsnotvalid1 deadpineapplem pmeaneyj3v9h pufzngc darkreddfly terryfunggg0xchaz 97282679101678 macyatmacy alexanderpannd7r8v peumingliu wangjunxintjwuz arunmurugan789tf3qu trasherdkx dnlowmanl yenyanjxano-1 qingygggjum5p yifeikong5 ellisgl1 sohokay94how 0uep3qj5c orngz akafaneh5ac0ch perellorodrigoshw4h corstianl izlisx christopyh caleballenw weiiavg janmalece jimbeamman7 sombocheayoror dm1korneev76har webspellerrxvub sls8h cyrelk8 onezhangxiaoyen rprragmicro edouard87jmvw ryohlann ant-cyjbtj4d agungtuanany9 domwebber3 agnipauf amartslpaka horselord-abrekqkhul aidenlin7 816397920532763 ridhofy tinaxdjll7r zhmvictoryh gindwyni sozonome5bm9x yvagsirkt 3378159108053 akafanehzcm5c italyqzn7v amirhbeigi7zch6f atilacamurcai starxitdyun hitrisrpshf jorgetolentinogmps8qu mitack6 cforney6 wangxiutingdychy ardieordenowttur irvanfza2 1091496 2deski4ix9d em1lyy3 cottomu saida93522c0sh0qu zlookd fieu1 jiavkuokons lifesweeteneryasit atrotter0zatt9z liu20800195spug bennyhano anyconhhv1 dattgoswami9lk5g edouard87jcn1b rgrxiao tjstephense realmark1r8h pbuzdinj littleyang0531m balloontmz6 tusharbihanis saida93522vochij yleklk-al guozhiming976 hoangtuan11046qu2qu luiguild2mvich dark15mfk4j 310222481154626 beleapryfey

min_max_similarity's Issues

Something about AF-SfMLearner

Hi, Sorry to interrupt. I can't find your contact information, so I'm asking a question here. I was recently reproducing AF-SfMLearner and I had the same problem as you, I predicted the trajectory very poorly. The author mentioned that you seem to have solved the problem, it seems to be by adding align_corner = True to F.interpolate and F.grid_sampler. Does it work? Thank you very much!

Best regards

What is the differences between Classifier and Projector?

I think the roles of the Classifier and Projector are quite similar. So why do you decide to differentiate them? Did you conduct any experiments with the Classifier and Projector sharing the same weights?

for each unlabeled image, is it be input to the different model with different data augmentation?

I am confused about contrast learning. My current understanding is that for models A and B, the same unlabeled image is enhanced with different data augmentation A1 and B1, and the corresponding data is put into models A and B respectively for subsequent similarity comparison. Am I right? If so, how to compute the L-similarity between the different augmentation(what if it is rotated or flipped, the loss could be high)

If my comprehension is totally wrong, could you tell me the correct procedure?
Thanks a lot.

Trying to reproduce, but extremely low performance

I'm trying to reproduce the reported results. I use the default hyper-parameters in train_mms.py. I wonder if anything need to change since I got extremely low results.

Train log
2022-10-03 20:00:16,393 2022-10-03 20:00:16.393250 Epoch [099/100], total_loss : 2.3719
2022-10-03 20:00:16,394 Train loss: 2.371871218389394
2022-10-03 20:00:28,635 Validation dice coeff model 1: 0.38830975438087945
2022-10-03 20:00:28,636 Validation dice coeff model 1: 0.3108561814208858
2022-10-03 20:00:28,637 current best dice coef model 1 0.47562526039001296, model 2 0.3603631227240354
2022-10-03 20:00:28,637 current patience :101

Test log
2022-10-03 20:11:16,730 logs/kvasir/test/saved_images_1/
2022-10-03 20:11:39,722 Model 1 F1 score : 0.18228444612672604
2022-10-03 20:11:39,838 Model 1 MAE : 0.18325799916872645
2022-10-03 20:11:39,839 logs/kvasir/test/saved_images_2/
2022-10-03 20:12:06,201 Model 2 F1 score : 0.15014741534272816
2022-10-03 20:12:06,324 Model 2 MAE : 0.1558560876074035

dataset

Thank you for your listening! May i ask you the url of all these datasets or share the datasets by some ways? Due to that for the first dataset i can not open the URL, for the second dataset, i can not get the dataset from the website, for the third dataset, i also can not open the URL.

truely thanks!

Encountering suboptimal results during the reproduction process, seeking your insights and guidance.

Hello, thank you for your contributions in the field of semi-supervised contrastive learning. I have gained a lot from your work!

However, during the reproduction process, I encountered some issues. When I attempted to apply the original model to the polyp dataset (kvasir-seg), I found that the performance was not ideal, with a dice score of only around 23. The log files also indicated that the dataloader filtered out many images with sizes not meeting the default trainsize.

Initially, I suspected the issue might be with the filters in data.py, so I commented out the filtering operations. This improved the training dice value to around 33, but it's clearly not reflective of the true capabilities of the model.

Next, I tried another polyp dataset, colon, and trained the original model. The best dice during training was over 85, which is quite impressive.

I then switched to the kvasir-instrument dataset mentioned in the original paper, using the original model parameters for training. However, the results were not satisfactory, similar to kvasir-seg, with a dice score of only around 20 during training.

What's strange is that the image sizes for all three datasets don't match the trainsize. I'm wondering if it's necessary to modify the default value of trainsize to try and improve experimental results?

This process has left me quite perplexed. I look forward to your response and greatly appreciate your assistance.