yeerwen / uniseg Goto Github PK

Thanks for you work. I want to ask how to copy Upstream/nnunet to replace nnunet. I have no idea.

Thanks for your work. I want to ask: as for my own several datasets and segmentation task, for example, to segment heart, liver, brain by several datasets, how to design my own segmentation task_id in code? Or may I ask which part of the relevant code is it?

The error originates from finish_online_evaluation: self.online_eval_tp = np.sum(self.online_eval_tp, 0) results in error as

setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (50,) + inhomogeneous part.

The list, self.online_eval_tp is accumulated in the run_iteration method of the UniSegTrainer class, by running run_online_evaluation. This function appends lists of varying sizes (based on the number of targets of the output parameter) to self.online_eval_tp (and the other lists). As described above, this cannot be summed over.

I believe this is due to the following:

• Each task (within MOTS, e.g. kidney, liver, lung, etc.) has different number of targets (3 or 2)
• ⁠The mapping between tasks to the number of outputs is fixed
• The code truncates the output of the model to match the task’s (fixed) specified number of channels
• ⁠This causes a misalignment when calculating the scores (different number of columns per task)
• ⁠Because the calculation of the number of columns is fixed, this will always result in an error (when using tasks with differing number of targets)

Thanks for your work, this helps me a lot, so I want to ask some question.

1. May I ask how to use the model trained on 11 upstream datasets to inference unlabeled data?
2. When I use the model trained on 11 upstream datasets to predict and inference unlabeled data, could I obtain the labels of all organs, tumors and vertebrae in these 11 upstream datasets?

Hello, I want to ask how to train upstream model on my own 2d data. I've tried to convert my own dataset to nnunet dataset. But it seems that I failed.

Hi! I found MOTS - Liver dataset only provide .nii format data. When I execute python Convert_MOTS_to_nnUNet_dataset.py, it will omit the Liver dataset. How to solve this problem? Could you provide the preprocessed liver dataset or the whole MOTS dataset?

Thank you for your answer!

Thansk for sharing your wonderful work.
Could you provide code for visualizing Error Map that suggested in Supplementary Materias if possible?
Waiting for your response. Thanks! :)

Traceback (most recent call last):
File "UniSeg_Metrics_test.py", line 115, in
evaluation(root, i)
File "UniSeg_Metrics_test.py", line 78, in evaluation
assert label.shape == pred.shape, [label.shape, pred.shape]
AssertionError: [(512, 512, 60), (498, 498, 300)]

我已经跑过你这个模型verse20的plan_and_preprocess，我想把预处理后的数据应用到其它模型，所以，这个模型预处理生成之后的文件是和nnunetv1的一模一样的吗？比如说这个DoDNetPlans_plans_3D是不是和nnUNetPlansv2.1_plans_3D一样

Your paper has been very helpful for our work, and when I read the code in UniSeg_trainer.py, I want to know how to set self.num_classes. Since many datasets have different classes, I noticed that you only set the num_classes for the output module seg_outputs.
self.seg_outputs.append(conv_op(self.conv_blocks_localization[ds][-1].output_channels, num_classes, 1, 1, 0, 1, 1, seg_output_use_bias))
Could you please explain how it works when dealing with datasets with varying numbers of classes?

Thank you for your excellent work. How do I train the uniseg please? nnUNet_train: error: unrecognized arguments: -exp_name UniSeg_Trainer

Hi,

Thanks for open-souring the amazing work. I am wondering if you can release the fine-tuning weight and preprocessing of your second place solution in MICCAI SegRap 2023? Thanks.

when I am running Python Convert in this step_ VerSe20_ To_ NnUNet_ Dataset. py has been stuck as shown in the picture and has not generated splits_ Final.pkl file

I'm highly interested in your work, but I may have limited browsing capabilities. I'm unsure about how to inject a prompt in nnUNet. (I only see two nnUNet folders.🤯)

Hi, I have two questions :)

1. In run_ssl.sh，there is only nnUNet_train, no nnUNet_predict. So how do I run inference? Will the nnUNet_train order automatically do the inference task?

2. I noticed you only provide UniSeg_Metrics_test.py on Upstream task. Is there a downstream version? How can I evaluate on downstream task?

really appreciate your help! looking forward to your reply.

When I apply the v1 preprocessed model to the new model, I find that Verse20's test is 0Is this normal？

Thank you for your excellent work. The idea of the task prompt is very innovative, but I am curious about how you design and get the ground truth of the target feature Ftp, because I found out you use output embedding instead of input embedding of the task-specific prompt. Could you please describe the task selection process?

I am doing the task of verse 2020, and I have completed the Move splits_ The final. pkl step, when I merge with Python_ Each_ Sub_ Dataet.py, it seems to default to Task091_ MOTS, how should I modify merge_ Each_ Sub_ Dataet.py makes it Task037_ VerSe20binary. Can I skip this step and train directly?

Thanks for your excellent work!

Due to limited computing resources, I would like to use the pre-trained model to inference directly, but I found that the supporting "plans.pkl" seems to be missing, can you release it if convenient?

Thanks again!

Hi! Hope everything is fine with you!

I'd like to train your model from scratch, but I feel training is a really long process. I've already trained for 3 days. I wonder how long is the total training time? (Upstream and Downstream, respectively). I'm using RTX 2080 ti (11G).

btw, happy new year :)

I first downloaded the dataset of B T C V, converted it to nnUNET format and preprocessed it, in addition to this error during training, what should I do, what is the problem.

/opt/conda/conda-bld/pytorch_1699449200967/work/aten/src/ATen/native/cuda/ScatterGatherKernel.cu:365: operator(): block: [1225,0,0], thread: [11,0,0] Assertion idx_dim >= 0 && idx_dim < index_size && "index out of bounds" failed.
/opt/conda/conda-bld/pytorch_1699449200967/work/aten/src/ATen/native/cuda/ScatterGatherKernel.cu:365: operator(): block: [1225,0,0], thread: [12,0,0] Assertion idx_dim >= 0 && idx_dim < index_size && "index out of bounds" failed.
/opt/conda/conda-bld/pytorch_1699449200967/work/aten/src/ATen/native/cuda/ScatterGatherKernel.cu:365: operator(): block: [1225,0,0], thread: [13,0,0] Assertion idx_dim >= 0 && idx_dim < index_size && "index out of bounds" failed.
/opt/conda/conda-bld/pytorch_1699449200967/work/aten/src/ATen/native/cuda/ScatterGatherKernel.cu:365: operator(): block: [1225,0,0], thread: [14,0,0] Assertion idx_dim >= 0 && idx_dim < index_size && "index out of bounds" failed.
/opt/conda/conda-bld/pytorch_1699449200967/work/aten/src/ATen/native/cuda/ScatterGatherKernel.cu:365: operator(): block: [1225,0,0], thread: [15,0,0] Assertion idx_dim >= 0 && idx_dim < index_size && "index out of bounds" failed.
/opt/conda/conda-bld/pytorch_1699449200967/work/aten/src/ATen/native/cuda/ScatterGatherKernel.cu:365: operator(): block: [1225,0,0], thread: [16,0,0] Assertion idx_dim >= 0 && idx_dim < index_size && "index out of bounds" failed.
/opt/conda/conda-bld/pytorch_1699449200967/work/aten/src/ATen/native/cuda/ScatterGatherKernel.cu:365: operator(): block: [1225,0,0], thread: [17,0,0] Assertion idx_dim >= 0 && idx_dim < index_size && "index out of bounds" failed.
/opt/conda/conda-bld/pytorch_1699449200967/work/aten/src/ATen/native/cuda/ScatterGatherKernel.cu:365: operator(): block: [1225,0,0], thread: [18,0,0] Assertion idx_dim >= 0 && idx_dim < index_size && "index out of bounds" failed.
/opt/conda/conda-bld/pytorch_1699449200967/work/aten/src/ATen/native/cuda/ScatterGatherKernel.cu:365: operator(): block: [1225,0,0], thread: [19,0,0] Assertion idx_dim >= 0 && idx_dim < index_size && "index out of bounds" failed.
/opt/conda/conda-bld/pytorch_1699449200967/work/aten/src/ATen/native/cuda/ScatterGatherKernel.cu:365: operator(): block: [1225,0,0], thread: [20,0,0] Assertion idx_dim >= 0 && idx_dim < index_size && "index out of bounds" failed.
/opt/conda/conda-bld/pytorch_1699449200967/work/aten/src/ATen/native/cuda/ScatterGatherKernel.cu:365: operator(): block: [1225,0,0], thread: [21,0,0] Assertion idx_dim >= 0 && idx_dim < index_size && "index out of bounds" failed.
Traceback (most recent call last):
File "/home/user_gou/anaconda3/envs/uniseg/bin/nnUNet_train", line 12, in
sys.exit(main())
File "/home/user_gou/anaconda3/envs/uniseg/lib/python3.9/site-packages/nnunet/run/run_training.py", line 179, in main
trainer.run_training()
File "/home/user_gou/anaconda3/envs/uniseg/lib/python3.9/site-packages/nnunet/training/network_training/nnUNetTrainerV2.py", line 455, in run_training
ret = super().run_training()
File "/home/user_gou/anaconda3/envs/uniseg/lib/python3.9/site-packages/nnunet/training/network_training/nnUNetTrainer.py", line 318, in run_training
super(nnUNetTrainer, self).run_training()
File "/home/user_gou/anaconda3/envs/uniseg/lib/python3.9/site-packages/nnunet/training/network_training/network_trainer.py", line 456, in run_training
l = self.run_iteration(self.tr_gen, True)
File "/home/user_gou/anaconda3/envs/uniseg/lib/python3.9/site-packages/nnunet/training/network_training/UniSeg_Trainer_DS.py", line 151, in run_iteration
l = self.loss(output, target)
File "/home/user_gou/anaconda3/envs/uniseg/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/user_gou/anaconda3/envs/uniseg/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/user_gou/anaconda3/envs/uniseg/lib/python3.9/site-packages/nnunet/training/loss_functions/deep_supervision.py", line 39, in forward
l = weights[0] * self.loss(x[0], y[0])
File "/home/user_gou/anaconda3/envs/uniseg/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/user_gou/anaconda3/envs/uniseg/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/user_gou/anaconda3/envs/uniseg/lib/python3.9/site-packages/nnunet/training/loss_functions/dice_loss.py", line 346, in forward
dc_loss = self.dc(net_output, target, loss_mask=mask) if self.weight_dice != 0 else 0
File "/home/user_gou/anaconda3/envs/uniseg/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/user_gou/anaconda3/envs/uniseg/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/user_gou/anaconda3/envs/uniseg/lib/python3.9/site-packages/nnunet/training/loss_functions/dice_loss.py", line 178, in forward
tp, fp, fn, _ = get_tp_fp_fn_tn(x, y, axes, loss_mask, False)
File "/home/user_gou/anaconda3/envs/uniseg/lib/python3.9/site-packages/nnunet/training/loss_functions/dice_loss.py", line 130, in get_tp_fp_fn_tn
tp = net_output * y_onehot
RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

Exception in thread Thread-5:
Traceback (most recent call last):
File "/home/user_gou/anaconda3/envs/uniseg/lib/python3.9/threading.py", line 980, in _bootstrap_inner
self.run()
File "/home/user_gou/anaconda3/envs/uniseg/lib/python3.9/threading.py", line 917, in run
self._target(*self._args, **self._kwargs)
File "/home/user_gou/anaconda3/envs/uniseg/lib/python3.9/site-packages/batchgenerators/dataloading/multi_threaded_augmenter.py", line 92, in results_loop
raise RuntimeError("One or more background workers are no longer alive. Exiting. Please check the print"
RuntimeError: One or more background workers are no longer alive. Exiting. Please check the print statements above for the actual error message
Exception in thread Thread-4:
Traceback (most recent call last):
File "/home/user_gou/anaconda3/envs/uniseg/lib/python3.9/threading.py", line 980, in _bootstrap_inner
self.run()
File "/home/user_gou/anaconda3/envs/uniseg/lib/python3.9/threading.py", line 917, in run
self._target(*self._args, **self._kwargs)
File "/home/user_gou/anaconda3/envs/uniseg/lib/python3.9/site-packages/batchgenerators/dataloading/multi_threaded_augmenter.py", line 92, in results_loop
raise RuntimeError("One or more background workers are no longer alive. Exiting. Please check the print"
RuntimeError: One or more background workers are no longer alive. Exiting. Please check the print statements above for the actual error message
已放弃 (核心已转储)

Thank you for your excellent work. I finished the training phase of upstream task and was trying to evaluate those models. But I noticed that all the pkl files created by Convert_XX_to_nnUNet_dataset contain empty 'test' dictionary. According to the Tab. 1 in paper, how can i find the test dataset of those upstream tasks? Thank you!

Thanks for great work.

"Note that the channel number of multi-scale segmentation maps is set to the maximum number of classes among all tasks."
it means that for output channel of decoder, it set to 3 (organ ,tumor, vertebra) ?
If I understand correctly the final output channel is 3 for all datasets?

It's great work. However I have some questions .
You mention "To accept the multi-modality inputs, we reform the first convolution layer of the model and set up three different convolution layers to handle the input with one, two, or four channels" .In my opinion it means that first layer and will be different in each different modalites number dataset case ,you replace first layer from these 3 different layers according input case.But Universial model aims at dealing all tasks with one model. How many channels you set up for nnUnet backbone ? How do you handle the different channel input, why not fix it 4 and replace lacking modalites as 0 ? Is Table 1,2 tested by one univerial model with all pretrain tasks?
Thanks very much!

Hello,

I noticed that you have a universal model named CLIP DoDnet. Does this involve creating a text representation for each dataset and then inputting it into the CLIP model, with other settings similar to DODnet?