kmaninis / cob Goto Github PK

Convolutional Oriented Boundaries

Home Page: http://www.vision.ee.ethz.ch/~cvlsegmentation/cob/

License: Other

MATLAB 36.12% Python 1.35% C++ 62.54%

cob's Introduction

COB: Convolutional Oriented Boundaries

COB is a technique that produces multiscale oriented contours and region hierarchies from a single forward pass of a generic image classification CNN. The whole pipeline runs in the order of 0.8 seconds per image to produce state-of-the-art hierarchical boundaries.

Check our visit our project page for pre-computed results (on BSDS500, PASCAL, SBD, MS-COCO, ImageNet, etc.), public benchmarks, results visualization, and additional information.

Update:

We have extended the ECCV paper with more experiments for RGB-D boundary detection, object boundary detection, as well as experiments that combine COB with high-level tasks that include Semantic Contours, Semantic Segmentation, Object Detection, and Segmented/Box Proposals. You can additionally download:

Model trained on NYUD-v2 train set (for RGB-D boundary detection) Link. All images and ground truth associated to the NYUD-v2 database can be found here. We followed the experimental of this paper.
Model trained on PASCAL Segmentation 2012 train set (for Object Boundary Detection) Link.
All additional results from our project page

Prerequisites:

Install this version of Caffe along with standard dependencies and matcaffe.
The CNN models of COB to reproduce the results of this paper are available. Download them and put them under models/:

Model trained on PASCALContext train set (VOC train) Link.
Model trained on PASCALContext trainval set (VOC trainval) Link.
Model trained on BSDS500 trainval set Link.

####Note: In the paper, we split the original val set of PASCALContext into VOC val and VOC test sets, while keeping the original training set as is (VOC train). VOC val was selected not to overlap with SBD val set.

COB Usage

To compute COB UCMs, see im2ucm.m (and scripts/im2cob_all.m)
To compute COB proposals, see im2prop.m (and scripts/im2cob_all.m)
For demos about how to use these functions, see the demos folder

For more information on how to use the code, please read README.txt

Citation:

If you use this code, please consider citing the following papers:

@Inproceedings{Man+16a,
  Title          = {Convolutional Oriented Boundaries},
  Author         = {K.K. Maninis and J. Pont-Tuset and P. Arbel\'{a}ez and L. Van Gool},
  Booktitle      = {European Conference on Computer Vision (ECCV)},
  Year           = {2016}
}

@Article{Man+17,
  Author 	 = {K.K. Maninis and J. Pont-Tuset and P. Arbel\'{a}ez and L. Van Gool},
  Title 	 = {Convolutional Oriented Boundaries: From Image Segmentation to High-Level Tasks},
  Journal 	 = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
  Year 		 = {2017}
}

If you encounter any problems with the code, please contact the first author.

Enjoy!

cob's People

Contributors

Stargazers

Watchers

Forkers

zhefan tybxiaobao rongchangzhao cv-ip cvml amiltonwong tjs2 yougoforward wenhuach stevenlol wanjinchang chagge caomw lyk125 benjamesbabala jlertle nrupatunga ranftlr zgsxwsdxg segmentationorg cmitash b2220333 weiliangxiao pkucactus mindosoft ankhoreva fireae donproc shuaifengzhi happog hhmaizi lejeunel jinlinyi farhodfm tpatten turgunyusuf khuang93 ybyangjing huberthomas dorahuiki pandinosaurus jamirando ssbagalkar robot520 hanyeliu jookuma ritvik06 robertwyq sudo-hannes tamnt027 biqiwhu gangmingzhao lkampoli technorior lu-david tiga002

cob's Issues

about the HED and CEDN scores

In the paper Convolutional Oriented Boundaries, you offered the hed and cedn scores(ODS) on PASCAL Context:

If it's convenient, could you offer your retrained model of these two method？or the predicted-images?（cause I cant get the result you presented in your paper） it will help me a lot，thank you！

compiling problems

I compiled COB on my computer, the environment is: Ubuntu14 with g++4.8.4.

The origin mex compile option is:
GCC='g++'
in build.m line64:

gcc_string = ['GCC=''' gcc_compiler ''' '];.

A error occured when I compile with this option which said:

COB/src/misc/containers.hpp:28:32: error: range-based ‘for’ loops
are not allowed in C++98 mode

Obviously c++11 is not on, so I add '-std=c++11' to the option:

gcc_string = ['GCC=''', gcc_compiler, ''' ', 'CXXFLAGS=''', '-std=c++11', ''' '];

Error resolved but another occurs:

mex_assess_one_sel.o: relocation R_X86_64_32 against
`.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC
/tmp/mex_2366922350995321_500/mex_assess_one_sel.o: error adding symbols: Bad value
collect2: error: ld returned 1 exit status

I don't exactly know what it is, but as it prompted, I go on add -fPIC to the CXXFLAGS:
gcc_string = ['GCC=''', gcc_compiler, ''' ', 'CXXFLAGS=''', '-std=c++11 -fPIC', ''' '];

everything works well after this, hope this experience helps somebody else.

How to get more proposals

Hi,
Thank you very much to release the code. I found that the number of proposals is smaller than EdgeBox. Is is possible to generate more proposals than your pre-computed ones? May I change some parameters to achieve this purpose? If so, which parameters are the most relevant ?
Thanks

models trained on COCO dataset

Hi
Thanks very much for the code, would you mind to release the model trained on COCO dataset ? I want to transfer it to my own dataset.

Resize input image to size bigger than 500x500

Hi, I've tried modifying the maxRes variable from 500 to 600 in cob_detect.m to resize my image to 600x600 but Matlab keeps crashing as soon as I run my script with the error:

syncedmem.hpp:30] Check failed: error == cudaSuccess (11 vs. 0)  invalid argument
*** Check failure stack trace: ***
Segmentation fault (core dumped)

Do I need to modify something in the models .prototxt or do you know if it is something related to the GPU memory? I'm running the script on a GTX 1080ti.

Typo on website : Retinal image underSANDING

Hi,

I just wanted to report that there seems to be a typo on the website http://people.ee.ethz.ch/~cvlsegmentation/

Or maybe it is a feature, not a bug, and I don't undersand it.

Cheers!

Started a pytorch port. Infos on training procedure

I am currently trying to reproduce the method on pyTorch. See here:
cobnet
FYI, a code (with python bindings) for the computation of the hierarchical merge tree is available in the higra package

I would therefore appreciate a more formal description of the training procedure as the paper is rather unclear on that matter.
From my understanding it goes like this.

phase 1:

Regress a bunch of dimensionality reduction (conv1x1) layer that connect to output of resNet blocks. The outputs you call "side activations" are 5 in total.
Regress two sets of parameters (conv1x1) for fine and coarse scales. These take as inputs tensors of 4 channels (concatenate 4 lowest and 4 highest side activations, respectively, to give Yfine and Ycoarse.
It is unclear what you mean by " To train the two sets of weights of the linear combinations, we freeze the pre-trained weights...".
Does that also apply to the side activation losses (eq. 1)? In that case, the parameters of the backbone would never be modified. My intuition is that phase 1 is in fact divided in two steps (1) sum all losses of eq. 1 and backprop through both dimensionality reduction and resNet. (2) freeze resNet and aforementioned dim reduction, compute sum of loss on Yfine and Ycoarse, backprop.
Can you confirm this please?

phase 2:

Freeze everything except the orientation layers to regress the oriented boundaries.

Thanks a lot in advance for your help.
Also, any help on the porting would be appreciated.

About segmentation benchmarking

Hi,
I'm new to this direction. And I have a few questions about how to benchmark on the NYUD-v2 dataset.

how do you get the segmentation groundtruth? I've downloaded the labeled data from NYUD-v2 website but there is no groundtruth but only 'labels' and 'instances' which may be used as groundtruth. But I don't know which one do you use in your paper.
What's more, if you use either 'labels' or 'instances', how do you dealing with the 'unlabeled' part(labeled as '0')?
The last question is that I find the size of 'ucm2' from you websize is '851x1121 double', which means segmentation result will be '425x560'. But resolutions of 'labels' and 'instances' are both standard '480x640' of Kinect.
Could you help me with the questions? Thank you very much.

Train the model

Hi,
Thank you very much for the code. Is also the code for train the model provided? I can't find it.

Undefined function

when I run im2ucm.m ,there is a error :
Undefined function 'loadvar' for input arguments of type 'char'.

bbox format

Question: The bbox coordinate of image's top-left is [0,0]. and the format of bbox is [y1, x1, y2, x2] corresponding to [up, left, down, right]. Is it right?

the resnet model w/o bn

Hi @kmaninis ,

thanks for your sharing code, and where could i found the res50 caffemodel w/o bn, or you just use the res50 model with BN initialize the weights? Thanks again, expecting for reply.

Getting error on running the file demo_cob_prop.m

When I ran the file demo_cob_prop I got an error

"Error using set_params (line 34)
Caffe path '/path/to/caffe/matlab/' not found"

Not sure which path I need to set and how. I just installed caffe locally for cpu.

Thanks

Demo output of demo_cob_prop.m

Hi guys,
I wanted to ask you if you (or someone else who made COB running) can verify the output of the demo program. I am not sure if the output "demo_cob_prop.m" produces the intended result. Compared to other examples it looks as some parts are missing which COB should capture (arms). I did not alter the code.

If you (someone) can maybe post the output of the demo, so that I am able to compare the results this would be great.

Thanks,
Markus

how to train the model

check failed: error == cudaSuccess(2 vs. 0) out of memory while testing

Hi,
I'm tring to run just 1 image (batch size: 1) by using COB_PASCALContext_trainval.caffemodel and deploy.prototxt provided in this repo, but got the out-of-memory error.

My hardware: Quadro P4000 with 8GB memories.

Does anyone know how to deal with this error? How many memories would COB use to run just 1 image? Thanks!

Appreciate any help

Output of COB

Hello, thank you very much for sharing the code. Can you share the test results of the COB object proposal? I want to judge whether the result of my run is correct?

matlab version