4uiiurz1 / pytorch-nested-unet Goto Github PK

The current IOU implementation has calculated the union and intersection over samples in a mini-batch, then it has calculated the IOU score. But I think IOU should be calculated per sample then take the mean over the samples in the mini-batch.

I think the following code is correct when the number of classes is 1:

intersection = (output_ & target_).sum(axis=(1, 2, 3))
union = (output_ | target_).sum(axis=(1, 2, 3))

return (intersection + smooth) / (union + smooth).mean()

I'm training a CNN and it looks like the program reads my dataset properly.

i = 0 
for f, breed in tqdm(df_train.values):
    if type(cv2.imread('train_{}.jpeg'.format(f)))==type(None):
        continue
    else:
        img = cv2.imread('train_{}.jpeg'.format(f))
        label = one_hot_labels[i]
        x_train.append(cv2.resize(img, (im_size1, im_size2)))
        y_train.append(label)
        i += 1
np.save('x_train2',x_train)
np.save('y_train2',y_train)
print('Done')

I then get the output which appended 35,126 images
100%|█████████████████████████████████████████████████████████████████████████| 35126/35126 [00:01<00:00, 25629.80it/s] Done
I then change the x_train and y_train to

y_train_raw = np.array(y_train, np.uint8)
x_train_raw = np.array(x_train, np.float32) / 255.

But when printing the shape it returns 0.

print(x_train_raw.shape)
print(y_train_raw.shape)

(0,) (0,)
Finally, when I try to split the dataset by calling this numpy...

X_train, X_valid, Y_train, Y_valid = train_test_split(x_train_raw, y_train_raw, test_size=0.1, random_state=1)

I get the error
ValueError: With n_samples=0, test_size=0.1 and train_size=0.9, the resulting train set will be empty. Adjust any of the aforementioned parameters.

Is it because I didn't append the dataset properly? Or did I not convert the images into tensors? If so, what would be the proper way to convert them into tensors?

As mentioned in the paper, how can I utilize the output of this code to retrieve single instances?

When I use "python train.py --dataset dsb2018_96 --arch NestedUNet" to run the code,why then it turns out"ValueError: With n_samples=0, test_size=0.2 and train_size=None, the resulting train set will be empty. Adjust any of the aforementioned parameters."?

When BatchNorm with affine=True is used, the previous layer's bias is cancelled (See the original BN paper for details). The current implementation's Conv2D layers are used with bias=True, so by changing bias=False, UNet and NestedUnet might be faster.

Is the mask label of the data trained by myself an 8-bit color label graph, or an 8-bit grayscale graph?

E:\Anaconda5.2\Anaconda\envs\tf2\python.exe "G:/Essay prep/pytorch-nested-unet-master/pytorch-nested-unet-master/train.py"
Traceback (most recent call last):
File "G:/pytorch-nested-unet-master/pytorch-nested-unet-master/train.py", line 301, in
main()
File "G:/pytorch-nested-unet-master/pytorch-nested-unet-master/train.py", line 218, in main
img_paths = glob('input/' + args.dataset + '/images/*')
TypeError: must be str, not NoneType
Config -----
name: None_NestedUNet_woDS
arch: NestedUNet
deepsupervision: False
dataset: None
input_channels: 3
image_ext: png
mask_ext: png
aug: False
loss: BCEDiceLoss
epochs: 10000
early_stop: 20
batch_size: 16
optimizer: Adam
lr: 0.0003
momentum: 0.9
weight_decay: 0.0001
nesterov: False

Process finished with exit code 1

I read your paper and the original of UNet++ and everything is clear to me apart from how I can I train the network using a new dataset.
There is the option of deep supervision in which the nework takes the co-registered image pairs (each has size 256x256x6) concatenated and as
output the [output1, output2, output3, output4], with each output dimension of 256x256x1.
I would like to reproduce your experiments with deep supervision.
It is clear to me what the input should be, but I fail to understand what the output should be. Specifically, how can I produce these 4 output matrices.
In the dataset from Lebedev only one grayscale image was used as output, and each pixel of this image was produced from subtracting the two input images.

@4uiiurz1 the model output is not segmented. It is simply blurry image. please help

My model output is not segmented image. It is blurry gray colored image. How to improve this. Thanks

Hi, how to get the other three layers' segmentation outputs when train with deep_supervision?

and why LovaszHingeLoss make the color of binary inference different?

Appreciate for any guidance!

thanks for your great working, but I can't find the pre traning model....

hi
How do I choose type of deep supervision? I can't see this in your code.
If I want to use type of ensemble for deep supervision how can I do? pruned too
thank you

If num_class default 1, mean mask contain of two color: white and black ?
You can explain argument deepsupervision. When do I should setup True for deepsupervision argument ?

after using the recommended "Import ''networks.tests.test_convert_pandas.pd "
showed ModuleNotFoundError: No module named 'pytest'

When I run the train.py,why the results are "loss nan - iou 0.0000 - val_loss nan - val_iou 0.0000"?I run on the dataset dsb2018_96.
loss nan - iou 0.0000 - val_loss nan - val_iou 0.0000
Epoch [93/100]
100%|███████████████████████████████████████████████████████████| 33/33 [00:01<00:00, 18.80it/s, loss=nan, iou=5.04e-10]
100%|█████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 36.14it/s, loss=nan, iou=5.76e-10]
loss nan - iou 0.0000 - val_loss nan - val_iou 0.0000
Epoch [94/100]
100%|███████████████████████████████████████████████████████████| 33/33 [00:01<00:00, 18.72it/s, loss=nan, iou=5.01e-10]
100%|█████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 36.95it/s, loss=nan, iou=5.76e-10]
loss nan - iou 0.0000 - val_loss nan - val_iou 0.0000
Epoch [95/100]
100%|███████████████████████████████████████████████████████████| 33/33 [00:01<00:00, 18.77it/s, loss=nan, iou=5.14e-10]
100%|█████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 36.63it/s, loss=nan, iou=5.76e-10]
loss nan - iou 0.0000 - val_loss nan - val_iou 0.0000
Epoch [96/100]
100%|███████████████████████████████████████████████████████████| 33/33 [00:01<00:00, 18.76it/s, loss=nan, iou=5.06e-10]
100%|█████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 36.84it/s, loss=nan, iou=5.76e-10]
loss nan - iou 0.0000 - val_loss nan - val_iou 0.0000
Epoch [97/100]
100%|███████████████████████████████████████████████████████████| 33/33 [00:01<00:00, 18.74it/s, loss=nan, iou=5.11e-10]
100%|█████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 37.12it/s, loss=nan, iou=5.76e-10]
loss nan - iou 0.0000 - val_loss nan - val_iou 0.0000
Epoch [98/100]
100%|███████████████████████████████████████████████████████████| 33/33 [00:01<00:00, 18.79it/s, loss=nan, iou=5.05e-10]
100%|█████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 36.90it/s, loss=nan, iou=5.76e-10]
loss nan - iou 0.0000 - val_loss nan - val_iou 0.0000
Epoch [99/100]
100%|████████████████████████████████████████████████████████████| 33/33 [00:01<00:00, 18.77it/s, loss=nan, iou=5.2e-10]
100%|█████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 37.04it/s, loss=nan, iou=5.76e-10]
loss nan - iou 0.0000 - val_loss nan - val_iou 0.0000

您好，为何模型不见了

Hi, thank you for this project,

I have a question about the dataset. Let's say if we have 20 classes, does it mean we should create 20 identical folders in the mask folder? I didn't understand this part. Thank you.

Hi, I'm sorry to disturb you. Do you have some recommendation of easy and multi-categories medical datasets for semantic segmentation.
I'm doing my undergraduate graduation project, and I'm lack of suitable dataset to do a multi-category semantic segmentation.

hi, i got a error 'CUDNN_STATUS_EXECUTION_FAILED' when run the test.py. Here is my environment: python3.6, cuda=9.0, cudnn=7.6.5, pytorch=0.4.0. I carefully followed the requirement.txt, but still got this error, any suggestion? Thank you

Hello, the code is awesome for that it uses some useful API for data preprocess(PyYAML and imgaug), can I get the dice output as well, since that there is definition for dice_coef but no output?
It got high iou score in gray mask but not with rgd , is there any way to edit ?
Is there any requirement for the image size?

With regards, any guidance would be so grateful!

hi everyone I need your help when I run my project I got this error I don't know how to solve
ValueError: With n_samples=0, test_size=0.2 and train_size=None, the resulting train set will be empty. Adjust any of the aforementioned parameters.
I don't know what kind of error or why this occurs can u please help me
Thank you

100%|██████████████████████████████████████████████████████████████| 1/1 [00:01<00:00, 1.95s/it, loss=0.671, iou=0.63]
100%|██████████████████████████████████████████████████████████| 1/1 [00:01<00:00, 1.71s/it, loss=0.788, iou=3.97e-10]
loss 0.6712 - iou 0.6299 - val_loss 0.7878 - val_iou 0.0000
=> early stopping

If the code is not modified, the same output file is created.

(output[i, c] * 255).astype('uint8'))

can anyone help??

是每个类别一个文件夹，每个类别一张图，该类别区域是255，其他区域是0

比如我有字母ABC三类，那么我需要3个文件夹，比如一张原图上面有AB两个字母，那么0文件夹的标签图是上只有A二值图，1文件夹标签图是上只有B二值图

是这样子吗

File "train.py", line 300, in
main()
File "train.py", line 206, in main
joblib.dump(args, 'models/%s/args.pkl' %args.name)
NameError: name 'joblib' is not defined
i m getting above error?

I used the model pruning technique to cut L4 to L2, iou from 0.8417 to 0.8391, but my time has increased, what is the reason? I only pruned the model in foward, no changes have been trained. pth file.

File "D:\pytorch-nested-unet-master\dataset.py", line 63, in getitem
img_id + self.mask_ext), cv2.IMREAD_GRAYSCALE)[..., None])
TypeError: 'NoneType' object is not subscriptable

i m getting this error while training model, Can anyone help??

Can you share the dataset with me? Thank you

Traceback (most recent call last):
File "train.py", line 367, in
main()
File "train.py", line 327, in main
train_log = train(config, train_loader, model, criterion, optimizer)
File "train.py", line 112, in train
for input, target, _ in train_loader:
File "/home/ubuntu/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 345, in next
data = self._next_data()
File "/home/ubuntu/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 856, in _next_data
return self._process_data(data)
File "/home/ubuntu/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 881, in _process_data
data.reraise()
File "/home/ubuntu/.local/lib/python3.6/site-packages/torch/_utils.py", line 395, in reraise
raise self.exc_type(msg)
TypeError: Caught TypeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/ubuntu/.local/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
data = fetcher.fetch(index)
File "/home/ubuntu/.local/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/ubuntu/.local/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/ubuntu/Desktop/pytorch-nested-unet-master/dataset.py", line 64, in getitem
mask.append(cv2.imread(os.path.join(self.mask_dir, str(i),img_id + self.mask_ext), cv2.IMREAD_GRAYSCALE)[..., None])
TypeError: 'NoneType' object is not subscriptable

Help!! I can't predict my model because it's giving me the error:
ValueError: Input 0 of layer dense_3 is incompatible with the layer: expected axis -1 of input shape to have value 100352 but received input with shape [None, 131072]

I just finished training a CNN using a ResNet50 architecture and a few top layers.
This is the code I used for creating the model...

base_model = ResNet50(weights = None, include_top=False, input_shape=(200, 200, 3))

x = base_model.output
x = Flatten()(x)
x = Dropout(0.2)(x)
x = Dense(32, activation='relu')(x)
x = Dense(16, activation='relu')(x)
predictions = Dense(num_class, activation='softmax')(x)

# The model to be trained
model = Model(inputs=base_model.input, outputs=predictions)

model.compile(loss='binary_crossentropy', 
              optimizer='rmsprop', 
              metrics=['accuracy'])

callbacks_list = [keras.callbacks.EarlyStopping(monitor='val_acc', verbose=1)]
model.summary()

As you can see, I only used a few layers on top of the ResNet50 architecture. I also used an image size input of 200, 200, 3
Went it calls back, this is the summary for the FLATTEN to the 2ND LAST DENSE LAYER.

__________________________________________________________________________________________________
flatten_1 (Flatten)             (None, 100352)       0           conv5_block3_out[0][0]           
__________________________________________________________________________________________________
dropout_1 (Dropout)             (None, 100352)       0           flatten_1[0][0]                  
__________________________________________________________________________________________________
dense_3 (Dense)                 (None, 32)           3211296     dropout_1[0][0]                  
__________________________________________________________________________________________________

The dense layer expects an input of shape 100,352 but instead it receives an input of 131,072!! Hence, the value error when running the predict code via...

img_path = 'train/10_right.jpeg'
img = image.load_img(img_path, target_size =(256,256))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)

preds = model.predict(x)

Could you tell me how to use your code for original datasets?
Thanks to your guide, I was able to use your code against 2018 Data Science Bowl dataset
So I tried to use the original dataset, but I couldn't.
Because there is more than one class to identify, I thought.
I tried to change the code, but I couldn't.
Please help me.

Thank you.

pc:~/unets/pytorch-nested-unet-master$ python train.py --dataset dsb2018_96 --arch NestedUNet
Config -----
name: dsb2018_96_NestedUNet_woDS
arch: NestedUNet
deepsupervision: False
dataset: dsb2018_96
input_channels: 3
image_ext: png
mask_ext: png
aug: False
loss: BCEDiceLoss
epochs: 10000
early_stop: 20
batch_size: 16
optimizer: Adam
lr: 0.0003
momentum: 0.9
weight_decay: 0.0001
nesterov: False

Traceback (most recent call last):
File "train.py", line 301, in
main()
File "train.py", line 222, in main
train_test_split(img_paths, mask_paths, test_size=0.2, random_state=41)
File "/home/jiyue/anaconda3/envs/judy/lib/python3.7/site-packages/sklearn/model_selection/_split.py", line 2100, in train_test_split
default_test_size=0.25)
File "/home/jiyue/anaconda3/envs/judy/lib/python3.7/site-packages/sklearn/model_selection/_split.py", line 1782, in _validate_shuffle_split
train_size)
ValueError: With n_samples=0, test_size=0.2 and train_size=None, the resulting train set will be empty. Adjust any of the aforementioned parameters

Nice code, thanks for sharing.

As you did not show results, in your opinion where nested-unet/unet++ stands in precision between standard resnet, densenet, squeezenet etc?

Does it perform much better?

我的dataset为：
data_name
——images
——masks

在加载数据集时报错：

<UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation> shows a high IoU of DSB2018 up to 92.37%.
This repo has any differences? Or do we need some tricks? have you think about it?
Hoping for reply. Thanks

Mat img = imread("test.png");
float scale = 1.0 / 255;
Size inputSize(256, 256);

Mat inputBolb = cv::dnn::blobFromImage(img, scale, inputSize, Scalar(), true, true);

dnn::Net my_net = cv::dnn::readNetFromONNX("unet_model.onnx");
my_net.setPreferableBackend(cv::dnn::DNN_BACKEND_OPENCV);
my_net.setPreferableTarget(cv::dnn::DNN_TARGET_CPU);

my_net.setInput(inputBolb);
Mat result = my_net.forward();

i try to deploy this model on opencv(after convert pth to onnx, two classes), it seems that result is single channel, so it can only get one label, but label is two classes, so how to get the two classes, any suggestioin?

please help

Where does the code reflect that "the global loss is weighted and accumulated by the loss functions of each branch"?