Sublabel-Accurate Convex Relaxations
Installation
To run the code, first install the convex optimization framework prost following the instructions presented there.
It is required to add some additional proximal and linear operators to prost. To do so, create the file CustomSources.cmake
in the directory prost/cmake/
with the following contents:
set(PROST_CUSTOM_SOURCES
"relative_path_to_sublabel_relax"/cvpr2016/prost/block_dataterm_sublabel.cu
"relative_path_to_sublabel_relax"/cvpr2016/prost/prox_ind_epi_polyhedral_1d.cu
"relative_path_to_sublabel_relax"/cvpr2016/prost/prox_ind_epi_conjquad_1d.cu
"relative_path_to_sublabel_relax"/cvpr2016/prost/block_dataterm_sublabel.hpp
"relative_path_to_sublabel_relax"/cvpr2016/prost/prox_ind_epi_polyhedral_1d.hpp
"relative_path_to_sublabel_relax"/cvpr2016/prost/prox_ind_epi_conjquad_1d.hpp
)
set(MATLAB_CUSTOM_SOURCES
"relative_path_to_sublabel_relax"/cvpr2016/prost/custom.cpp
)
Replace "relative_path_to_sublabel_relax"
with the relative path to go from the directory prost/cmake
to the directory where you cloned this repository into (e.g., ../../sublabel_relax
).
After adding this file, recompile prost again, e.g., run in the directory prost/build
cmake ..
make -j16
Finally, before running the MATLAB scripts the mex-File for computing convex envelopes has to be compiled. In the directory of this repository, run from within MATLAB the following command:
mex compute_convex_conjugate.cpp
Finally, add the folder prost/matlab
to your MATLAB path, e.g. by writing
addpath('~/Documents/Projects/prost/matlab')
from within MATLAB.
Usage
The code for reproducing individual numerical experiments from the paper are organized in different files.
ROF denoising, Figure 5:
rof_direct.m
Direct optimization of the ROF model without functional liftingrof_baseline.m
Optimization of the ROF model with the baseline approachrof_sublabel.m
Sublabel-accurate optimization of the ROF model
Denoising with robust truncated quadratic dataterm, Figure 6:
truncrof_baseline.m
Optimzation using the baseline approachtruncrof_sublabel.m
Sublabel-accurate version
Stereo matching, Figure 9:
stereo_baseline.m
Baseline approach for stereo matchingstereo_sublabel.m
Sublabel-accurate approach
Running the sublabel-stereo experiment with 4 labels should then produce the following expected output
>> stereo_sublabel
[compute_convex_conjugate] Total number of slopes: 5025269
[compute_convex_conjugate] Original points: 50017500 (Reduction factor 1.00e-01).
prost v0.2-build-2016-06-17
Running on device number 0: GeForce GTX TITAN X (12.0 GB, 3072 Cores).
# primal variables: 2223000
# dual variables: 4816500
Memory requirements: 255MB (11858/12287MB available).
It 1: Feas_p=6.09e+02, Eps_p=2.80e-02, Feas_d=0.00e+00, Eps_d=1.49e-02;
It 1043: Feas_p=3.21e+00, Eps_p=4.09e-02, Feas_d=1.76e-02, Eps_d=1.49e-02;
It 2085: Feas_p=7.98e-01, Eps_p=4.09e-02, Feas_d=1.68e-02, Eps_d=1.49e-02;
It 3126: Feas_p=2.25e-01, Eps_p=4.09e-02, Feas_d=1.63e-02, Eps_d=1.49e-02;
It 4168: Feas_p=7.54e-02, Eps_p=4.09e-02, Feas_d=1.51e-02, Eps_d=1.49e-02;
It 4825: Feas_p=3.97e-02, Eps_p=4.09e-02, Feas_d=1.49e-02, Eps_d=1.49e-02;
Reached convergence tolerance.
Elapsed time is 24.236633 seconds.
Publications
The following publications describe the approach:
-
Sublabel-Accurate Relaxation of Nonconvex Energies (T. Möllenhoff, E. Laude, M. Moeller, J. Lellmann, D. Cremers), In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
-
Sublabel-Accurate Convex Relaxation of Vectorial Multilabel Energies (E. Laude, T. Möllenhoff, M. Moeller, J. Lellmann, D. Cremers), In European Conference on Computer Vision and Pattern Recognition (ECCV), 2016.