miou of cityscapes about pytorch-segmentation HOT 9 CLOSED

This is what I get with the xception65_cityscapes_trainfine checkpoint on cityscapes val split. The results are slightly lower than official TF model (80,42%). I suspect this difference to be related to issue #10 .
The evaluation was done with the official Cityscapes evaluation script.

from pytorch-segmentation.

@remtav Thanks. Your experiment result is obtained in cityscapes val dataset?

from pytorch-segmentation.

@yazhe2017 Yes! I edited my post accordingly. Thanks!

from pytorch-segmentation.

hi! Can you tell me how to use the official Cityscapes evaluation script to eval this model?Thanks a lot for your help!

from pytorch-segmentation.

This is what I get with the xception65_cityscapes_trainfine checkpoint on cityscapes val split. The results are slightly lower than official TF model (80,42%). I suspect this difference to be related to issue #10 .
The evaluation was done with the official Cityscapes evaluation script.

Why is there Nan and what is nIou?

from pytorch-segmentation.

is there Nan and what is nIou?

As can be seen in the details for each submission to cityscapes-dataset.com (ex.: https://www.cityscapes-dataset.com/method-details/?submissionID=1366), nIoU is instance IoU and only gives a result for categories with instance annotation (ex.: person 1, person 2, person 3, etc.).

from pytorch-segmentation.

from pytorch-segmentation.

Sorry for late reply.
I fixed issue #10 and added code to evaluate cityscapes.
The results are as follows:

classes          IoU      nIoU
--------------------------------
road          : 0.984      nan
sidewalk      : 0.866      nan
building      : 0.931      nan
wall          : 0.626      nan
fence         : 0.635      nan
pole          : 0.668      nan
traffic light : 0.698      nan
traffic sign  : 0.800      nan
vegetation    : 0.929      nan
terrain       : 0.651      nan
sky           : 0.954      nan
person        : 0.832    0.645
rider         : 0.644    0.452
car           : 0.956    0.887
truck         : 0.869    0.420
bus           : 0.906    0.657
train         : 0.834    0.555
motorcycle    : 0.674    0.404
bicycle       : 0.783    0.605
--------------------------------
Score Average : 0.802    0.578
--------------------------------


categories       IoU      nIoU
--------------------------------
flat          : 0.988      nan
construction  : 0.937      nan
object        : 0.729      nan
nature        : 0.931      nan
sky           : 0.954      nan
human         : 0.842    0.667
vehicle       : 0.944    0.859
--------------------------------
Score Average : 0.904    0.763
--------------------------------

from pytorch-segmentation.

thanks a lot for you ! I'll try it again.

from pytorch-segmentation.