Code Monkey home page Code Monkey logo

automaticpanoramicimagestitching-autopanostitch's Introduction

AutoPanoStitch

View Automatic panorama stitcher (AutoPanoStitch) on File Exchange Open in MATLAB Online

Automatic Panorama Stitching software in MATLAB. Spherical, cyclindrical and planar projections stitching is supported in this version and can recognize multiple panoramas.

Stitched images 1:

Type Images
Stitched image pano_full
Crop box pano_bbox
Cropped image pano_crop

Stitched images 2:

Type Images
Stitched image result_26
Cropped image cropped

Requirements

MATLAB
MATLAB Computer Vision Toolbox
MATLAB Image Processing Toolbox
MATLAB Parallel Computing Toolbox
MATLAB Optimization Toolbox

Run command

Please use the Main_AutoPanoStitch.m to run the program. Change the folderPath = '../../../Data/Generic'; to your desired folder path. Also, change the folderName = ''; to a valid name. You can download the whole Generic folder datasets in AutoPanoStitch Stitching Datasets Compilation.

Change the hyper parameters accordingly if needed. But it is not required though.

%% Inputs 2
%--------------------------------------------------------------------------
% Parallel workers
input.numCores = str2double(getenv('NUMBER_OF_PROCESSORS'));    % Number of cores for parallel processing
input.poolType = 'numcores';     % 'numcores' | 'Threads'

%% Inputs 3
% Warping
input.warpType = 'planar';   % 'spherical' | 'cylindrical' | 'planar' (projective)

% Focal length
input.fx = 2000;       % focal length of camera in pixels
input.fy = 2000;       % focal length of camera in pixels

% Distortion coefficients [k1, k2, k3, p1, p2]
input.DC = [0, 0, 0, 0, 0];

% Feature matching
input.detector = 'SIFT';                % 'HARRIS' | 'SIFT' | 'vl_SIFT' | 'FAST' | 'SURF' | 'BRISK' | 'ORB' | 'KAZE'
input.Matchingthreshold = 3.5;          % 10.0 or 1.0 (default) | percent value in the range (0, 100] | depends on 
                                        % binary and non-binary features
input.Ratiothreshold = 0.6;             % ratio in the range (0,1]
input.Sigma = 1.6;                      % Sigma of the Gaussian (1.4142135623)
input.NumLayersInOctave = 4;            % Number of layers in each octave -- SIFT only
input.ContrastThreshold = 0.00133;      % Contrast threshold for selecting the strongest features, 
                                        % specified as a non-negative scalar in the range [0,1]. 
                                        % The threshold is used to filter out weak features in 
                                        % low-contrast regions of the image. -- SIFT only
input.EdgeThreshold = 6;                % Edge threshold, specified as a non-negative scalar greater than or equal to 1. 
                                        % The threshold is used to filter out unstable edge-like features  -- SIFT only  
input.nearestFeaturesNum = 5;           % Nearest images minimum number of features to filter
                                        % distant image matches (filter gain overlap images to reduce complexity)

% Image matching (RANSAC)
input.Matchingmethod = 'Approximate';   %'Exhaustive' (default) | 'Approximate'
input.Inliersconfidence = 99.9;         % Inlier confidence [0,100]
input.maxIter = 2000;                   % RANSAC maximum iterations
input.Transformationtype = 'affine';     % 'rigid' | 'similarity' | 'affine' | 'projective' | 'translation'
input.MaxDistance = 1.50;               % Maximum distance (pixels) 1.5

% Image blending and panorama
input.gainDerivation = 1;           % Two types of gain matrix derivations 1 or 2 (both leads to same results with some roundoffs)
input.sigmaN = 10;                  % Standard deviations of the normalised intensity error
input.sigmag = 0.01;                % Standard deviations of the gain
input.resizeImage = 1;              % Resize input images
input.resizeStitchedImage = 1;      % Resize stitched image
input.maxPanoramaArea = 3e6;        % Maximum panorama area
input.blending = 'multiband';       % 'multiband' | 'linear' | 'none'
input.bands = 6;                    % bands
input.MBBsigma = 5;                 % Multi-band Gaussian sigma
input.filtSize = [5,5];             % Gaussian kernel Filter size
input.parforSummation = true;       % Gain diagonal elements summation by parfor
input.showPanoramaImgsNums = 0;     % display the panorama images with numbers after tranform 0 or 1

% Post-processing
input.canvas_color = 'black';       % Panorama canvas color 'black' | 'white'
input.showCropBoundingBox = 1;      % Display cropping bounding box 0 | 1
input.blackRange = 0;               % Minimum dark pixel value to crop panaroma
input.whiteRange = 250;             % Minimum bright pixel value to crop panaroma
input.showKeypointsPlot  = 0;       % Display keypoints plot (parfor suppresses this flag, so no use)
input.displayPanoramas = 0;         % Display panoramas in figure

Note

Depending on how your images are captured, panaroma being a spherical, cylindrical or planar should be selected judicially using the input.warpType and input.Transformationtype. Generally, spherical or cylindrical projections with affine or rigid transformation should work well in most of the cases. However, some panoramas specifically looks good in projective transformation, e.g. flatbed scanner or whiteboard (affine works well too) images.

Currently, spherical, cyclindrical and planar projections stitching is supported in this version and can recognize multiple panoramas. This work is in progress, further improvements such as the inclusion of a full view 360 x 180-degree panoramas stitching (everything visible from a point), automatic panorama straightening, runtime speed optimization and Graphical User Interface (GUI) are under development. Your patience will be appreciated.

Image stitching/panorama datasets

Creating image stitching datasets takes a lot of time and effort. During my Ph.D. days, I tried to compile datasets that were comprehensive to have spherical, cylindrical or planar and full view 360 x 180-degree panoramas. These datasets posed a real challenge to the automatic stitching method. If all these datasets are stitched well, it definitely shows the robustness of your stitching method.

All these datasets are public! Some of them were from my Ph.D. studies (especially on cracks) and most of them were downloaded from the internet. I do not remember the individual names of the dataset providers. But I acknowledge their work and I am thankful to all of them! I hope you appreciate their efforts in making these datasets public to advance the research!

Below are some samples from the datasets. There are 98 panorama or image stitching/registration datasets in total. You can download them in AutoPanoStitch Stitching Datasets Compilation. If I come across any interesting and challenging datasets, I will expand this compilation.

Type Images
CMU dataset_samples_CMU0
Grand Canyon dataset_samples_grandcanyon
Shanghai dataset_samples_shanghai
UCSB dataset_samples_ucsb4
Yellowstone dataset_samples_yellowstone
Rio dataset_samples_rio

Citation

Image stitching datasets for cracks are available to the public. If you use this specific dataset (related to cracks) in your research, please use the following BibTeX entry to cite:

@PhdThesis{preetham2021vision,
author = {{Aghalaya Manjunatha}, Preetham},
title = {Vision-Based and Data-Driven Analytical and Experimental Studies into Condition Assessment and Change Detection of Evolving Civil, Mechanical and Aerospace Infrastructures},
school =  {University of Southern California},
year = 2021,
type = {Dissertations & Theses},
address = {3550 Trousdale Parkway Los Angeles, CA 90089},
month = {December},
note = {Condition assessment, Crack localization, Crack change detection, Synthetic crack generation, Sewer pipe condition assessment, Mechanical systems defect detection and quantification}
}

Known issues

  1. Bundle adjustment using the lsqnonlin in MATLAB is computationally slow.
  2. Panoramic images needs to be straightened, some of the panoramas will be skewed.

Adaptation of open source

Bundle adjustment functions in MATLAB are adapted from the Kevin Luo's GitHub Repo and heavily improved.

Licensing conditions

The original implementation of the automatic panaroma stitching by Dr. Brown was written in C++ and is LICENSED under The University of British Columbia. This is being programmed and made available to public for academic / research purposes only. Please cite the relevant citations as provided in the main file.

Acknowledgements

To all the authors who made the image stitching datasets public.

Feedback

Please rate and provide feedback for the further improvements.

automaticpanoramicimagestitching-autopanostitch's People

Contributors

preethamam avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

automaticpanoramicimagestitching-autopanostitch's Issues

the gain compensation values of the two images obtained from the file gainCompensation. m are equal

Hello, I read your code recently. When I use two images for stitching, I find that the gain compensation values of the two images obtained from the file gainCompensation. m are equal. When stitching multiple images, the gain compensation values of each image are different. In theory, when stitching two images, the gain compensation value should also be different. Can you test whether there is the problem I said?

Error while using your amazing code

Hallo Sir,

I am getting this error while running your main file, why is this happening ?, What should I do to fix it, I am totally new in MATLAB.

Error:
Starting parallel pool (parpool) using the 'Processes' profile ...
Error using parpool
Too many workers requested. The cluster "Processes" has the NumWorkers property set to a maximum of 14 workers but 32 workers were requested. Either request a number of workers less than NumWorkers, or
increase the value of the NumWorkers property for the cluster (up to a maximum of 512 for the Local cluster).

Error in Main_AutoPanoStitch (line 36)
parpool(input.numCores);

Thanks a lot in advanced.

Can not find detectSIFTFeatures function?

Main_AutoPanoStitch (line 62)
[keypoints, allDescriptors, images, imageinfo, imageFocals, numImg]
= featureMatching(input, imgSetVector, myImg);

featureMatching>(parfor body) (line 44)
[descriptors, points] = getFeaturePoints(input, image);

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.