Code Monkey home page Code Monkey logo

pytorch-segmentation's Introduction

PytorchSegmentation

This repository implements general network for semantic segmentation.
You can train various networks like DeepLabV3+, PSPNet, UNet, etc., just by writing the config file.

DeepLabV3+

Pretrained model

You can run pretrained model converted from official tensorflow model.

DeepLabV3+(Xception65+ASPP)

$ cd tf_model
$ wget http://download.tensorflow.org/models/deeplabv3_cityscapes_train_2018_02_06.tar.gz
$ tar -xvf deeplabv3_cityscapes_train_2018_02_06.tar.gz
$ cd ../src
$ python -m converter.convert_xception65 ../tf_model/deeplabv3_cityscapes_train/model.ckpt 19 ../model/cityscapes_deeplab_v3_plus/model.pth

Then you can test the performance of trained network.

$ python eval_cityscapes.py --tta

mIoU of cityscapes

$ pip install cityscapesScripts
$ export CITYSCAPES_RESULTS=../output/cityscapes_val/cityscapes_deeplab_v3_plus_tta
$ export CITYSCAPES_DATASET=../data/cityscapes
$ csEvalPixelLevelSemanticLabeling 
classes          IoU      nIoU
--------------------------------
road          : 0.984      nan
sidewalk      : 0.866      nan
building      : 0.931      nan
wall          : 0.626      nan
fence         : 0.635      nan
pole          : 0.668      nan
traffic light : 0.698      nan
traffic sign  : 0.800      nan
vegetation    : 0.929      nan
terrain       : 0.651      nan
sky           : 0.954      nan
person        : 0.832    0.645
rider         : 0.644    0.452
car           : 0.956    0.887
truck         : 0.869    0.420
bus           : 0.906    0.657
train         : 0.834    0.555
motorcycle    : 0.674    0.404
bicycle       : 0.783    0.605
--------------------------------
Score Average : 0.802    0.578
--------------------------------


categories       IoU      nIoU
--------------------------------
flat          : 0.988      nan
construction  : 0.937      nan
object        : 0.729      nan
nature        : 0.931      nan
sky           : 0.954      nan
human         : 0.842    0.667
vehicle       : 0.944    0.859
--------------------------------
Score Average : 0.904    0.763
--------------------------------

MobilenetV2

$ cd tf_model
$ wget http://download.tensorflow.org/models/deeplabv3_mnv2_cityscapes_train_2018_02_05.tar.gz
$ tar -xvf deeplabv3_mnv2_cityscapes_train_2018_02_05.tar.gz
$ cd ../src
$ python -m converter.convert_mobilenetv2 ../tf_model/deeplabv3_mnv2_cityscapes_train/model.ckpt 19 ../model/cityscapes_mobilnetv2/model.pth

How to train

In order to train model, you have only to setup config file.
For example, write config file as below and save it as config/pascal_unet_res18_scse.yaml.

Net:
  enc_type: 'resnet18'
  dec_type: 'unet_scse'
  num_filters: 8
  pretrained: True
Data:
  dataset: 'pascal'
  target_size: (512, 512)
Train:
  max_epoch: 20
  batch_size: 2
  fp16: True
  resume: False
  pretrained_path:
Loss:
  loss_type: 'Lovasz'
  ignore_index: 255
Optimizer:
  mode: 'adam'
  base_lr: 0.001
  t_max: 10

Then you can train this model by:

$ python train.py ../config/pascal_unet_res18_scse.yaml

Dataset

Directory tree

.
├── config
├── data
│   ├── cityscapes
│   │   ├── gtFine
│   │   └── leftImg8bit
│   └── pascal_voc_2012
│        └── VOCdevkit
│            └── VOC2012
│                ├── JPEGImages
│                ├── SegmentationClass
│                └── SegmentationClassAug
├── logs
├── model
└── src
    ├── dataset
    ├── logger
    ├── losses
    │   ├── binary
    │   └── multi
    ├── models
    └── utils

Environments

  • OS: Ubuntu18.04
  • python: 3.7.0
  • pytorch: 1.0.0
  • pretrainedmodels: 0.7.4
  • albumentations: 0.1.8

if you want to train models in fp16

  • NVIDIA/apex: 0.1

Reference

Encoder

Decoder

SCSE

IBN

OC

PSP

ASPP

pytorch-segmentation's People

Contributors

moridaiki avatar nyoki-mtl avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pytorch-segmentation's Issues

can not convert mobilenetv2

The command which convert the pretrained model(MobilenetV2) to pytorch is mistake.
Maybe correct command is
$ python -m converter.convert_mobilenetv2 ../tf_model/deeplabv3_mnv2_cityscapes_train/model.ckpt 19 ../model/cityscapes_mobilnetv2/model.pth

And, this error is occurred

FileNotFoundError: [Errno 2] No such file or directory: '../model/mobilenetv2_encoder/model.pth'

I solve it, change src/models/net.py line 92 pretrained=False.

Is it OK?

I think there is a wrong in Focalloss.py

You write the focalloss just like this:
image
By doing this, the logpt is a scale not a tensor. If the batchsize is greater than 1, I think it is wrong here.
An I right?

Cityscapes classes IoU 0.802 train from scratch or use pretrained model?

Hello,
I find pretrained model is not included in cityspcae yaml file.Is the best result training from scratch? Or use the the model convert from tenosrflow imagenet pretrained model, I find in your training code:

if pretrained_path:

when you load the pretrained model, the model network includes some layers not in xception65. Is your loading model converting from tenosrflow cityspace pretrained model, or training from scratch?
look froward to you reply, thanks

miou of cityscapes

Hi, thanks for the great repo. Could you report the miou of cityscapes for deeplab v3+? Thanks.

How's the training result?

Thank you for sharing your codes!

Is there any other config that can reproduce the paper result, since I stuck at best valid iou 0.47 when I tried to train it on on Cityscapes using DeepLabV3+(default config)?

Thanks again and appreciate your help!

ImportError at converting the pretrained model

Im getting the following ImportError when converting the pretrained model.

python convert.py ../tf_model/deeplabv3_cityscapes_train/model.ckpt 19 ../model/cityscapes_deeplab_v3_plus/model.pth

Traceback (most recent call last):
File "convert.py", line 6, in
from models.net import SPPNet
File "/scratch-local/cdtemp/christian/pytorch-segmentation/src/models/net.py", line 6, in
from .encoder import create_encoder
File "/scratch-local/cdtemp/christian/pytorch-segmentation/src/models/encoder.py", line 3, in
from torchvision import models
ImportError: cannot import name 'models'

Problem solved using " conda install torchvision -c pytorch"

freeze BN on single GPU

Thank you for your great repo. I noticed that you didn't freeze the BN layer when training on single GPU. If I train on cityscape, do I need to freeze BN with single GPU and small batch_size?

Normalization to [-1, 1] for DeepLab

Hi @nyoki-mtl,

I would like to ask a quick question please.

I see in the code that, for DeepLab, you normalize the input image to the range [-1, 1] like this:

image = (image/255)*2 - 1

Could you please tell me how you came up with this? I did some experiments and this normalization indeed yields the best performance. I checked the official TensorFlow implementation but I couldn't find where they used this normalization.

Thank you in advance for your response!

num_samples=0

When I do
python eval_cityscapes.py --tta
I got:
num_samples should be a positive integer value, but got num_samples=0
Any suggestion?
Thank you!

TTA Method in DeeplabV3+ | Evaluation

What is the intension of the TTA method? Does it deliver better results?

I want to use this network as an input to another network and im not sure, whether I should use the model.tta method.

with torch.no_grad():
    for batched in valid_loader:
        images, labels = batched
        images_np = images.numpy().transpose(0, 2, 3, 1)
        labels_np = labels.numpy()

        images, labels = images.to(device), labels.to(device)
        preds = model.tta(images, net_type='deeplab')
        preds = preds.argmax(dim=1)
        preds_np = preds.detach().cpu().numpy()

        images_list.append(images_np)
        labels_list.append(labels_np)
        preds_list.append(preds_np)

        if len(images_list) == 4:
            break

Question about prediction

Let's say I want to use the pre-trained DeepLabV3+ model on an incomplete image of the Cityscapes dataset (i.e. an image with an hole in the middle). Is there a way to do not predict the pixel in the "hole"?

prediction failure on Cityscapes images

While testing the code with supplied Cityscapes config file and pretrained weights, about 1 prediction out of 80 fails completely. See attached pictures. Did not occur on original tensorflow model.

191_img_2019-02-28
191_pred_2019-02-28_color

212_img_2019-02-28
212_pred_2019-02-28_color

017_img_2019-02-28
017_pred_2019-02-28_color

097_img_2019-02-28
097_pred_2019-02-28_color

Discrepancy in performance compared to paper

Hi,

First of all, thanks for the code! I was looking for a PyTorch port of DeepLabv3+ that can use the official TensorFlow checkpoints. Your repo is the only one I could find! Great work!

There seems however to be some discrepancy compared to the TensorFlow version.
I tried the xception65_coco_voc_trainval checkpoint on the Pascal VOC test set with flipping and multi-scale ([0.5:0.25:1.75]) inference, and obtained mIoU = 85.93630 (by uploading to the evaluation server), which far inferior to the reported 87.80%.

In addition to the difference in BatchNorm's eps that you discovered previously, there should be some other details that affect the performance. Unfortunately I was unable to find them. I hope you or somebody can.

How to serialize the model for libtorch?

Thanks for your contribution to the community. I want to use C++ with libtorch and Cuda to load the referred model (Xception65+ASPP), but errors occured in the loading process. Is it because the model is not a standard pytorch model?

The code is shown as follow:
#include

#include "torch/script.h"

int main(int argc, const char* argv[]) {
const std::string module_path = "./model/cityscapes_deeplab_v3_plus/model.pt";
torch::jit::script::Module module;

try{
    module = torch::jit::load(module_path);
    std::vector<torch::jit::IValue> inputs;
    inputs.emplace_back(torch::rand({1, 3, 1024, 2048}).to(torch::kCUDA));
    at::Tensor output = module.forward(inputs).toTensor();
}catch (std::exception& e){
   std::cout << e.what();
}
return 0;

}

Here is the log in the terminal:
The following operation failed in the TorchScript interpreter.
Traceback of TorchScript, serialized code (most recent call last):
File "code/torch/models/net.py", line 14, in forward
_1 = self.decoder
_2 = self.spp
_3, _4, _5, _6, _7, _8, = (self.encoder).forward(input, )
~~~~~~~~~~~~~~~~~~~~~ <--- HERE
_9 = (_1).forward(_4, _5, _6, _7, (_2).forward(_3, ), _8, )
return (_0).forward(_9, )
File "code/torch/models/xception.py", line 60, in forward
_25 = (_22).forward((_23).forward(_24, ), )
_26 = (_23).forward1((_21).forward(_25, ), )
_27 = (_19).forward((_20).forward(_26, ), )
~~~~~~~~~~~~ <--- HERE
_28, _29, _30, _31, _32, _33, = _27
_34 = (_17).forward((_18).forward(_28, ), )
File "code/torch/models/xception/___torch_mangle_46.py", line 17, in forward
_3 = self.sep_conv2
_4 = (self.sep_conv1).forward(argument_1, )
_5 = (_3).forward(_4, )
~~~~~~~~~~~ <--- HERE
_6 = (_2).forward(_5, )
_7 = (_0).forward((_1).forward(argument_1, ), )
File "code/torch/models/common/___torch_mangle_38.py", line 8, in forward
def forward(self: torch.models.common.___torch_mangle_38.SeparableConv2d,
argument_1: Tensor) -> Tensor:
return (self.block).forward(argument_1, )
~~~~~~~~~~~~~~~~~~~ <--- HERE
File "code/torch/torch/nn/modules/container/___torch_mangle_37.py", line 16, in forward
_2 = self.bn_depth
_3 = (self.depthwise).forward((self.relu).forward(argument_1, ), )
_4 = (_0).forward((_1).forward((_2).forward(_3, ), ), )
~~~~~~~~~~~ <--- HERE
return _4
File "code/torch/torch/nn/modules/container/___torch_mangle_37.py", line 15, in forward
_1 = self.pointwise
_2 = self.bn_depth
_3 = (self.depthwise).forward((self.relu).forward(argument_1, ), )
~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
_4 = (_0).forward((_1).forward((_2).forward(_3, ), ), )
return _4
File "code/torch/torch/nn/modules/conv/___torch_mangle_33.py", line 8, in forward
def forward(self: torch.torch.nn.modules.conv.___torch_mangle_33.Conv2d,
argument_1: Tensor) -> Tensor:
input = torch._convolution(argument_1, self.weight, None, [1, 1], [1, 1], [1, 1], False, [0, 0], 256, False, False, True, True)
~~~~~~~~~~~~~~~~~~ <--- HERE
return input

Traceback of TorchScript, original code (most recent call last):
/home/tony/Programs/miniconda3/envs/torch_env/lib/python3.8/site-packages/torch/nn/modules/conv.py(419): _conv_forward
/home/tony/Programs/miniconda3/envs/torch_env/lib/python3.8/site-packages/torch/nn/modules/conv.py(423): forward
/home/tony/Programs/miniconda3/envs/torch_env/lib/python3.8/site-packages/torch/nn/modules/module.py(709): _slow_forward
/home/tony/Programs/miniconda3/envs/torch_env/lib/python3.8/site-packages/torch/nn/modules/module.py(725): _call_impl
/home/tony/Programs/miniconda3/envs/torch_env/lib/python3.8/site-packages/torch/nn/modules/container.py(117): forward
/home/tony/Programs/miniconda3/envs/torch_env/lib/python3.8/site-packages/torch/nn/modules/module.py(709): _slow_forward
/home/tony/Programs/miniconda3/envs/torch_env/lib/python3.8/site-packages/torch/nn/modules/module.py(725): _call_impl
/media/tony/D/Projects/python_ws/pseg/src/models/common.py(32): forward
/home/tony/Programs/miniconda3/envs/torch_env/lib/python3.8/site-packages/torch/nn/modules/module.py(709): _slow_forward
/home/tony/Programs/miniconda3/envs/torch_env/lib/python3.8/site-packages/torch/nn/modules/module.py(725): _call_impl
/media/tony/D/Projects/python_ws/pseg/src/models/xception.py(30): forward
/home/tony/Programs/miniconda3/envs/torch_env/lib/python3.8/site-packages/torch/nn/modules/module.py(709): _slow_forward
/home/tony/Programs/miniconda3/envs/torch_env/lib/python3.8/site-packages/torch/nn/modules/module.py(725): _call_impl
/media/tony/D/Projects/python_ws/pseg/src/models/xception.py(123): forward
/home/tony/Programs/miniconda3/envs/torch_env/lib/python3.8/site-packages/torch/nn/modules/module.py(709): _slow_forward
/home/tony/Programs/miniconda3/envs/torch_env/lib/python3.8/site-packages/torch/nn/modules/module.py(725): _call_impl
/media/tony/D/Projects/python_ws/pseg/src/models/net.py(18): forward
/home/tony/Programs/miniconda3/envs/torch_env/lib/python3.8/site-packages/torch/nn/modules/module.py(709): _slow_forward
/home/tony/Programs/miniconda3/envs/torch_env/lib/python3.8/site-packages/torch/nn/modules/module.py(725): _call_impl
/home/tony/Programs/miniconda3/envs/torch_env/lib/python3.8/site-packages/torch/jit/_trace.py(934): trace_module
/home/tony/Programs/miniconda3/envs/torch_env/lib/python3.8/site-packages/torch/jit/_trace.py(733): trace
eval.py(91):
RuntimeError: CUDA out of memory. Tried to allocate 128.00 MiB (GPU 0; 5.80 GiB total capacity; 3.99 GiB already allocated; 164.31 MiB free; 4.01 GiB reserved in total by PyTorch)

Process finished with exit code 0

ERROR:ZeroDivisionError: division by zero

how can i slove this problem when i run python train.py ../config/pascal_unet_res18_scse.yaml?
Traceback (most recent call last):
File "train.py", line 159, in
loss = loss_fn(preds, labels)
File "/home/yirumeng/anaconda3/envs/Pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 547, in call
result = self.forward(*input, **kwargs)
File "/home/yirumeng/code/CAM/pytorch-segmentation-master/src/losses/multi/init.py", line 26, in forward
loss = self.criterion(preds, labels)
File "/home/yirumeng/anaconda3/envs/Pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 547, in call
result = self.forward(*input, **kwargs)
File "/home/yirumeng/code/CAM/pytorch-segmentation-master/src/losses/multi/lovasz_loss.py", line 77, in forward
total_loss += lovasz_softmax_flat(prb, lbl, self.ignore_index, self.only_present)
File "/home/yirumeng/code/CAM/pytorch-segmentation-master/src/losses/multi/lovasz_loss.py", line 56, in lovasz_softmax_flat
return total_loss / cnt
ZeroDivisionError: division by zero

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.