Hi,

Can I use this directly on images to get results ?

If I download the Semantic Segmentation Models will it work ?

Thanks

Thanks for your code.The following is the result after I implemented the command :bash run_cocostuff.sh.I don't kown where is wrong and how to solve it.Can you help me?

layer {
name: "drop7_2"
type: "Dropout"
bottom: "fc7_2"
top: "fc7_2"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "fc8_cocostuff_2"
type: "Convolution"
bottom: "fc7_2"
top: "fc8_cocostuff_2"
convolution_param {
num_output: 172
kernel_size: 1
}
}
layer {
name: "fc6_3"
type: "Convolution"
bottom: "pool5"
top: "fc6_3"
convolution_param {
num_output: 1024
pad: 18
kernel_size: 3
dilation: 18
}
}
layer {
name: "relu6_3"
type: "ReLU"
bottom: "fc6_3"
top: "fc6_3"
}
layer {
name: "drop6_3"
type: "Dropout"
bottom: "fc6_3"
top: "fc6_3"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "fc7_3"
type: "Convolution"
bottom: "fc6_3"
top: "fc7_3"
convolution_param {
num_output: 1024
kernel_size: 1
}
}
layer {
name: "relu7_3"
type: "ReLU"
bottom: "fc7_3"
top: "fc7_3"
}
layer {
name: "drop7_3"
type: "Dropout"
bottom: "fc7_3"
top: "fc7_3"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "fc8_cocostuff_3"
type: "Convolution"
bottom: "fc7_3"
top: "fc8_cocostuff_3"
convolution_param {
num_output: 172
kernel_size: 1
}
}
layer {
name: "fc6_4"
type: "Convolution"
bottom: "pool5"
top: "fc6_4"
convolution_param {
num_output: 1024
pad: 24
kernel_size: 3
dilation: 24
}
}
layer {
name: "relu6_4"
type: "ReLU"
bottom: "fc6_4"
top: "fc6_4"
}
layer {
name: "drop6_4"
type: "Dropout"
bottom: "fc6_4"
top: "fc6_4"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "fc7_4"
type: "Convolution"
bottom: "fc6_4"
top: "fc7_4"
convolution_param {
num_output: 1024
kernel_size: 1
}
}
layer {
name: "relu7_4"
type: "ReLU"
bottom: "fc7_4"
top: "fc7_4"
}
layer {
name: "drop7_4"
type: "Dropout"
bottom: "fc7_4"
top: "fc7_4"
dropout_param {
dropout_ratio: 0.5
}
}
layer {
name: "fc8_cocostuff_4"
type: "Convolution"
bottom: "fc7_4"
top: "fc8_cocostuff_4"
convolution_param {
num_output: 172
kernel_size: 1
}
}
layer {
name: "fc8_cocostuff"
type: "Eltwise"
bottom: "fc8_cocostuff_1"
bottom: "fc8_cocostuff_2"
bottom: "fc8_cocostuff_3"
bottom: "fc8_cocostuff_4"
top: "fc8_cocostuff"
eltwise_param {
operation: SUM
}
}
layer {
name: "fc8_interp"
type: "Interp"
bottom: "fc8_cocostuff"
top: "fc8_interp"
interp_param {
zoom_factor: 8
}
}
layer {
name: "fc8_mat"
type: "MatWrite"
bottom: "fc8_interp"
include {
phase: TEST
}
mat_write_param {
prefix: "cocostuff/features/deeplabv2_vgg16/val/fc8/"
source: "cocostuff/list/val_id.txt"
strip: 0
period: 1
}
}
layer {
name: "silence"
type: "Silence"
bottom: "label"
bottom: "data_dim"
}
I0616 15:55:01.469554 6570 layer_factory.hpp:77] Creating layer data
I0616 15:55:01.469599 6570 net.cpp:106] Creating Layer data
I0616 15:55:01.469609 6570 net.cpp:411] data -> data
I0616 15:55:01.469641 6570 net.cpp:411] data -> label
I0616 15:55:01.469656 6570 net.cpp:411] data -> data_dim
I0616 15:55:01.469676 6570 image_seg_data_layer.cpp:46] Opening file cocostuff/list/val.txt
I0616 15:55:01.470676 6570 image_seg_data_layer.cpp:68] A total of 1000 images.
F0616 15:55:01.510185 6570 syncedmem.hpp:18] Check failed: error == cudaSuccess (30 vs. 0) unknown error
*** Check failure stack trace: ***
@ 0x7fbaaa9dadaa (unknown)
@ 0x7fbaaa9dace4 (unknown)
@ 0x7fbaaa9da6e6 (unknown)
@ 0x7fbaaa9dd687 (unknown)
@ 0x7fbaab106ca8 caffe::SyncedMemory::mutable_cpu_data()
@ 0x7fbaab115f0c caffe::Blob<>::Reshape()
@ 0x7fbaab1163a9 caffe::Blob<>::Reshape()
@ 0x7fbaab00c95a caffe::ImageSegDataLayer<>::DataLayerSetUp()
@ 0x7fbaab08c3d3 caffe::ImageDimPrefetchingDataLayer<>::LayerSetUp()
@ 0x7fbaab159005 caffe::Net<>::Init()
@ 0x7fbaab15a758 caffe::Net<>::Net()
@ 0x406bf7 test()
@ 0x4059dc main
@ 0x7fbaa9ce5f45 (unknown)
@ 0x406111 (unknown)
@ (nil) (unknown)
run_cocostuff.sh: line 82: 6570 Aborted (core dumped) ${CMD}
hdl1@hdl1:~/cocostuff/models/deeplab-public-ver2$

Hi
I have annotated json files of each image made through (labelme) tool how should I convert this into cocostuff dataset formt?

Should it be 183 or 172?

Hi, @nightrome , I appreciate for your works, it really helpful. But for fair comparison, I have to evaluate the models on the older COCO_stuff dataset, i.e., vision 1.0. I have no idea about the full class list of it which only contains 172 classes. Can you help me to get these information? Thanks a lot.

Hi all,

While I was using annotator to convert my own annotations to DeepLab format, I failed to do so.

Here is what I did,

1. I followed the procedure and ran the Matlab command "CocoStuffAnnotator()". I was able to annotate my own image and saved changes.
2. I then ran the Matalb command "convertAnnotationsDeeplab()". But Matlab gave me an error message like this:

   Reference to non-existent field 'S'.
     Error in convertAnnotationsDeeplab (line 26)
     labelMap = labelStruct.S;

3. Since I did not change any line of code in Matlab scripts, I was thinking maybe something went wrong with my image file. Then I compared the example annotated image file and then applied my own annotation to that file. Interestingly, the results are different.
   Here is the labelStruct content of file annotated by myself:

COCO_train2014_000000439995_2.mat
      drawSizeMap: [427×640 double]
         imageIdx: 2
        imageName: 'COCO_train2014_000000439995'
        imageSize: [427 640 3]
         labelMap: [427×640 double]
       labelNames: {94×1 cell}
      timeDiffMap: [427×640 double]
        timeImage: 19.6003
    timeImageDraw: 0
          timeMap: [427×640 double]
        timeTotal: 6.4350
         userName: 'example'

Here is the labelStruct content of default annotated file:

COCO_train2014_000000439995.mat
                    S: [427×640 double]
             captions: {5×1 cell}
                names: {182×1 cell}
    regionLabelsStuff: [925×1 double]
       regionMapStuff: [427×640 double]

Clearly, the output of CocoStuff annotator does not contain field "S", which could fail the Matalb command "convertAnnotationsDeeplab(). Why is that?

Thanks in advance!

I download the cocostuff-10k-v1.1.zip. when i extract it, error occured.
Could you help check it?

Is cocostuff available in the coco json annotation format?

I would like to use this stuff annotation tool to annotate another dataset(like coco-format)
but how can I generate data/input/regions/***.mat ?

Matlab is generating following error upon executing "demo_cocoStuff();" after "startup();"

Undefined function or variable 'imageInsertBlobLabels'.
Error in demo_cocoStuff (line 52)
labelMapThingsIm = imageInsertBlobLabels(labelMapThingsIm, labelMapThings, labelNames);

Additional paths are required in startup.m for specifying location of extra source files in Matlab 2015b.

addpath('dataset/code/utils');
addpath('dataset/code/conversion');

Hello
I want to load this dataset by using pycocotools but I got this problem

And I am quite confused about some part of the file 'cocostuff-10k-v1.1.json'

Hi I am trying to convert the provided json file cocostuff-10k-v1.1.json to t7 format using coco api available here. I am using the lua function coco.CocoApi("cocostuff-10k-v1.1.json") to do this. Iam getting an error
convert: cocostuff-10k-v1.1.json --> .t7 [please be patient] ...ri/mnt_hari/torch/install/share/lua/5.1/coco/CocoApi.lua:142: Expected comma or array end but found T_OBJ_BEGIN at character 104320833 stack traceback: [C]: in function 'decode' ...ri/mnt_hari/torch/install/share/lua/5.1/coco/CocoApi.lua:142: in function '__convert' ...ri/mnt_hari/torch/install/share/lua/5.1/coco/CocoApi.lua:128: in function '__init' ...hari/mnt_hari/torch/install/share/lua/5.1/torch/init.lua:91: in function <...hari/mnt_hari/torch/install/share/lua/5.1/torch/init.lua:87> [C]: in function 'CocoApi' [string "_RESULT={coco.CocoApi("cocostuff-10k-v1.1.jso..."]:1: in main chunk [C]: in function 'xpcall' ...hari/mnt_hari/torch/install/share/lua/5.1/trepl/init.lua:661: in function 'repl' ...hari/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:204: in main chunk [C]: at 0x00405d50 [1.4312s]
while running this. Is there a way to do this?

It cannot access gasonMex.CPP file even the path is given

Hi , for saving time, could you share the trained Deeplab models(resnet101, vgg16)?

Thanks for your great works. But I have questions : 1. Could this tool make bboxes of objects? 2. Could this tool make captions of single image? 3. Should all parts be labeled as some names?
Thanks a lot.

In Annotation tool,
While doing this step: Extract the thing annotations for all images in Matlab: extractThings()
I get error on Matlab
Loading and preparing annotations... Undefined function or variable 'gasonMex'
Did anyone had similar issue or know how to fix it?
Thank you!

Hello! I download COCO dataset for training dataset on YOLO. But annotations turns out on .json format. How i convert them to YOLO? YOLO have a .txt labels .

Thanks, and sorry if my question is stupid :-)

Hi，

First of all, thanks for your great work.

My question is, after I finished training my model, the training process gave me some results for my test set. Those results are in .mat format. Is there any script that I can use to visualize .mat files (e.g., a script that help me convert .mat file into .png file with different color for different class)? I have been searching a lot but I am not a Matlab guru.

Thanks in advance!

hi~
I download the code of SLIC_mex.
i run the code of SLIC_mex successfully.
but how to generate data/input/regions/***.mat?

my software:matlabR2016a

Hello after running the run_cocostuff_vgg16.sh script, I am getting this error :

Could not open or find file cocostuff/data/annotations/COCO_train2014_000000034795.png

It happens because all files' formats are .mat under annotations folder. I think this step doing it but I don't have any MATLAB experience and knowledge.

Convert the annotations by running the Matlab script: startup(); convertAnnotationsDeeplab();

thank you for all helps.

Hello,

i downloaded the dataset but to only realize that the annotations folder has .mat files instead of png files.

how can i covert these .mat files to png ?

i plan to train them on pytorch deeplab v3

Hii

The annotation for segmentation in cocostuff-10k-v1.1.json is a list. ie
"segmentation":[ { "counts":xxxxxx, "size":[x,y] } ]
How ever the output produced by convertAnnotationsJSON.py does not contain this . The corresponding code just has
anndata['segmentation'] = Rs
which will produce this
"segmentation": { "counts":xxxxxx, "size":[x,y] }

Is this a bug or my understanding is wrong ?

Is the normal annotation (coco) of things not included in the JSON of stuff? As in how would i get the category of every object on an image and not just the category other for all the thing classes?

Thanks for your great work, but I met this following problems:

extractThings
Loading COCO API...
Loading and preparing annotations... DONE (t=11.14s).
Error using containers.Map/subsref
The specified key is not present in this container.

Error in CocoApi/getAnnIds (line 121)
t = coco.loadAnns(coco.inds.imgAnnIdsMap(imgIds));

Error in extractThings>getImLabelMap (line 88)
annIds = cocoApi.getAnnIds('imgIds', imgId, 'iscrowd', []);

Error in extractThings (line 71)
labelMap = getImLabelMap(cocoApi, image, imageName);

Could u tell me what's wrong here? I use new dataset. And I want to create new annotation of new classes. Thanks a lot.