Code Monkey home page Code Monkey logo

opencvswiftstitch's Introduction

OpenCV computer vision with iOS: stitching panoramas

Version 5.0 of OpenCVStitch - updated for Swift 5.0 + OpenCV 4.3
See appropriate branches and tags for Swift 2.x - 4.x

This project was created to a answer a couple of Stack Overflow questions:
libraries to CAPTURE panorama in iOS
Can I mix Swift with C++? Like the Objective - C .mm files

v2 demonstrates how to mix Swift, Objective-C and C++ in one project whilst keeping the code clearly separate.

The project AppDelegate and View Controller are written in Swift. Swift cannot talk directly to C++ (which we need for OpenCV), so we provide an Objective-C++ wrapper class to mediate between Swift and C++. We also provide an Objective-C++ category on UIImage to mediate between UIImage and CV::Mat image formats. The CVWrapper header file is pure Objective-C. For v1(which doesn't use Swift) this separation was a matter of clean style. For v2, it is a requirement: if any C++ headers are included in the wrapper, the app will not compile (Swift won't like it).

Installation
To run the project you need to install the OpenCV framework using Cocoapods

Assuming you have first installed CocoaPods, run 'pod install' in this directory to install OpenCV for the project. From then on, always open the project in XCode from the SwiftStitch.xcworkspace file that the pod install creates.

The podfile will install openCV v4.3.0.

Use
OpenCVStitch is a very simple iOS/openCV example showing basic use of the Stitcher class. The c++ code is adapted from a sample included with the openCV distribution.

The app has almost no user interface. On launch, the stitching code operates on four sample images, displaying the result in a UIScrollView.

OpenCVSwiftStitch Versions
Version 5.1
Swift / Objective-C / C++
OpenCV 4.3
Modern Swift - async / await
Error, exception handling from C++ -> Objective-C -> Swift
Tested with XCode 13.2 / Swift 5.0 for iOS 13.0+
Version 5.0
Swift / Objective-C / C++
OpenCV 4.3
Tested with XCode 12.0 / Swift 5.0 for iOS 9.0+
Version 4.0
Swift / Objective-C / C++
Tested with XCode 10.0 / Swift 4.2 for iOS 8.0+
Version 3.0
Swift / Objective-C / C++
Tested with XCode 8.0 / Swift 3.0 for iOS 8.0+
Version 2.1
Swift / Objective-C / C++
Tested with XCode 7.0 / Swift 2.0 for iOS 7.0+
Version 2.0
Swift / Objective-C / C++
Tested with XCode 6.4 / Swift 1.2 for iOS 7.0+
Version 1.0
Objective-C / C++
Tested with XCode 4.5.2 -> 6.3 for iOS 5.1 upwards

Provides a partial answer to: Libraries to capture panoramas in iOS 6 (Stack Overflow)

OpenCV Versions
OpenCV 4.3
The podfile installs OpenCV 4.3

this version of OpenCVStitch opted to use cocoapods to overcome the notorious installation issues with previous versions of the framework

XCode 12

Project is now updated for Swift 5.0 and XCode 12. For backwards compatibility checkout the 2.0 / 2.1 branches, 3.0 /4.0 releases or refer to the Objective-C version v1.0.

Comparisons

OpenCV 2.4.9OpenCV 3.0.0OpenCV 3.1.0OpenCV 4.3.0
v2_inset2 v3_inset2 v3.1_inset2 v5_inset2
v2_inset2 v2_inset2 v2_inset2 v5_inset2

opencvswiftstitch's People

Contributors

foundry avatar rinatkhanov avatar robmurrer avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

opencvswiftstitch's Issues

Infinite scrolling

Sorry for the question again! Do you think there is an easy way to implement horizontal infinite scrolling? I am tinkering with scrollViewDidEndDecelerating and can determine that I have reached the right side of the pano, but is there a way to wrap the left side of the pano to the right side and vice versa? Thanks!

About using vector<vector<Rect>> and steps to use this library

First, Great Library! I must thank you for your effort @foundry !

I've read both foundry/OpenCVStitch#1 and foundry/OpenCVStitch#2 on your another opencv lib the one with objc, and I have started a project that I met those problems before I read them.
With foundry/OpenCVStitch#2 I guess he bypassed the problem by saving the pictures first to the filesystem and then read it back and stitch them. I've drawn the same conclusion and tested successfully on my side as well. My question is why? Why saving to the filesystem and then reinitiate back to UIImage can solve this problem? I've also noticed that if we don't do that (saving to the filesys first) the memory usage when stitching happening will double itself everything it tries to stitch a one more picture in the array.

And is it the same we can do this img1+img2 = result_image and then we do result_image+img3 = result_image and then result_image+img4 and go on? Is it the same to do img1+img2+img3+img4 together? The result will be the same?

My last question and the most important one I think is, I hope you can give me a direction as how may I use this instead.

Stitcher::stitch(InputArray images, 
                                const std::vector<std::vector<Rect>>& rois, 
                                OutputArray pano)

I think I should modify your obj-c wrapper to include this. Problem is I tried several ways to send in vectors and I need to send in this parameter from swift to objc and from objc to c++, and I get all the compiling issues. I wonder if you can give me a simple line of code or at least point me a direction of how to pass in this parameter from swift.

OpenCV 3

It seems that when downloading the 3.0.0 framework from their site, it downloads the same framework. How did you get?

Linker problems, and some questions

Hi! I'm trying to learn about OpenCV-iOSSwift when I found your project. First of all thanks for developing it. However I have a problem when I try to build it this:

ld: library not found for -lPods
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Searching on Internet It could be a Build Phases/"Link Binary with Libraries" problem, however in this section there is an entry called "libPods.a" and I thougth it's the same library named in two different forms.

So there my questions:
1)Where to change Pods to libPods, in order to fix this bug?
2)How should I change the project to use the bridge with the newest version of OpenCV 3.0 framework, I have already downloaded? Does this change avoid to use pods?

Thanks for you attention.

Doesn't currently compile, Linker errors

I'm excited to learn OpenCV and your project looked like a good place to get started. After running pod install, I tried to compile the app and ran in to linker errors. I'm not used to working with C++. Do you know how to get past these errors?

Using OpenCV (2.4.10)

ERRORS:

Undefined symbols for architecture arm64:
"_jpeg_free_small", referenced from:
_free_pool in opencv2(jmemmgr.o)
_self_destruct in opencv2(jmemmgr.o)
"_jpeg_mem_available", referenced from:
_realize_virt_arrays in opencv2(jmemmgr.o)
"_jpeg_free_large", referenced from:
_free_pool in opencv2(jmemmgr.o)
"_jpeg_open_backing_store", referenced from:
_realize_virt_arrays in opencv2(jmemmgr.o)
"_jpeg_get_large", referenced from:
_alloc_large in opencv2(jmemmgr.o)
_alloc_barray in opencv2(jmemmgr.o)
"_jpeg_mem_term", referenced from:
_jinit_memory_mgr in opencv2(jmemmgr.o)
_self_destruct in opencv2(jmemmgr.o)
"_jpeg_mem_init", referenced from:
_jinit_memory_mgr in opencv2(jmemmgr.o)
"_jpeg_get_small", referenced from:
_jinit_memory_mgr in opencv2(jmemmgr.o)
_alloc_small in opencv2(jmemmgr.o)
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Several Swift Compiler Error without any code change

Hi Foundry,

Thank you so much for putting up a demo for many of us who have just landed in the world of iOS and Swift and have no ideas what's going on.

I git cloned the project and did the pod install and it successfully installed the opencv.

But after I clicked the Build and Run button, it throws me 5 errors without any code change.

image

Could you please help?

My System Specs:

MacOS: Catalina 10.15.7
XCode: 12.4
Swift: 5
Checked out branch: master

Thank you.

Import problems for some users

A number of users, myself included, have found that including this code in their own programs will cause a run-time error with the string "cvflann::anyimpl::bad_any_cast".

This can be fixed with two identical edits in stitching.h and UIImage+OpenCV.h. Simply change the line that reads:

#import <opencv2/opencv.hpp>

to:

#import <opencv2/stitching.hpp>
#import <opencv2/imgcodecs.hpp>

It appears there is a problem in another part of the OpenCV build, and simply importing just the parts you need avoids it. I didn't think this was worth a pull, but I'd be happy to if you think it's useful.

OpenCV2/opencv.hpp not found

I hate to be asking such a simple question, but I seem to be getting tripped up by something rather trivial -- I keep getting an "opencv2/opencv.hpp" is not found error. What I've done so far:
1)Downloaded your proj. and opened in XCode
2)Downloaded OpenCV 3.0 from the link you supplied
3)Dragged the framework into the Frameworks folder of the project
4)Commented out the lines of stitching.cpp according to your instruction above.
5)Noticed that OpenCV was already in a "required" state in the build phase for the project.

What'd I miss?

i'm getting an out of memory crash when stitching 3 images (320px*640px)

stitching...
malloc: *** mach_vm_map(size=2606923776) failed (error code=3)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
OpenCV Error: Insufficient memory (Failed to allocate 2606919364 bytes) in OutOfMemoryError, file /Users/kyle/code/opensource/opencv/modules/core/src/alloc.cpp, line 52
OpenCV Error: Assertion failed (u != 0) in create, file /Users/kyle/code/opensource/opencv/modules/core/src/matrix.cpp, line 424
libc++abi.dylib: terminating with uncaught exception of type cv::Exception: /Users/kyle/code/opensource/opencv/modules/core/src/matrix.cpp:424: error: (-215) u != 0 in function create

Pinch Zoom

Are you able to pinch zoom with this? I have not been successful yet. I see that you have

 self.scrollView.maximumZoomScale = 4.0
 self.scrollView.minimumZoomScale = 0.5

So I figured there might be the ability to zoom. Thanks again!

Crashes on iPhone 6+ w/ iOS 10.1 beta

The program works great when using either the pre-built or my build of the OpenCV 3.1 framework in the simulator (which is iOS 10.1 14B70).

HOWEVER, I’m getting a crash on iOS 10.1 (14B72) when** I run OpenCVSwiftStich on the device after downloading and compiling (with the python script) OpenCV 3.1 from the GitHub head.

The crash is in flann::getParam()...unfortunately I don’t really know C++ so I’m kinda useless here. Also OpenCV is some of the ugliest code I’ve seen in any language, ever.

If I try downloading the pre-built OpenCV 3.1 framework and using it on the device with iOS 10.1 I get link errors for missing symbols.

-W

Here’s the crash on 10.1:

#0  0x0000000184bceefc in __cxa_throw ()
#1  0x00000001002fbfa0 in cvflann::flann_algorithm_t cv::flann::getParam<cvflann::flann_algorithm_t>(cv::flann::IndexParams const&, cv::String const&, cvflann::flann_algorithm_t const&) ()
#2  0x00000001002fbad0 in cv::flann::Index::build(cv::_InputArray const&, cv::flann::IndexParams const&, cvflann::flann_distance_t) ()
#3  0x00000001002fba50 in cv::flann::Index::Index(cv::_InputArray const&, cv::flann::IndexParams const&, cvflann::flann_distance_t) ()
#4  0x00000001002ed42c in cv::FlannBasedMatcher::train() ()
#5  0x00000001002e7e54 in cv::DescriptorMatcher::knnMatch(cv::_InputArray const&, std::__1::vector<std::__1::vector<cv::DMatch, std::__1::allocator<cv::DMatch> >, std::__1::allocator<std::__1::vector<cv::DMatch, std::__1::allocator<cv::DMatch> > > >&, int, cv::_InputArray const&, bool) ()
#6  0x00000001002e7bf4 in cv::DescriptorMatcher::knnMatch(cv::_InputArray const&, cv::_InputArray const&, std::__1::vector<std::__1::vector<cv::DMatch, std::__1::allocator<cv::DMatch> >, std::__1::allocator<std::__1::vector<cv::DMatch, std::__1::allocator<cv::DMatch> > > >&, int, cv::_InputArray const&, bool) const ()
#7  0x0000000100470d74 in (anonymous namespace)::CpuMatcher::match(cv::detail::ImageFeatures const&, cv::detail::ImageFeatures const&, cv::detail::MatchesInfo&) ()
#8  0x000000010046ea10 in cv::detail::BestOf2NearestMatcher::match(cv::detail::ImageFeatures const&, cv::detail::ImageFeatures const&, cv::detail::MatchesInfo&) ()
#9  0x000000010046e5bc in (anonymous namespace)::MatchPairsBody::operator()(cv::Range const&) const ()
#10 0x000000010027d410 in (anonymous namespace)::block_function(void*, unsigned long) ()
#11 0x000000010102d778 in _dispatch_client_callout2 ()
#12 0x00000001010412e8 in _dispatch_apply_invoke_and_wait ()
#13 0x0000000101040c18 in dispatch_apply_f ()
#14 0x000000010027d37c in cv::parallel_for_(cv::Range const&, cv::ParallelLoopBody const&, double) ()
#15 0x000000010046e1d0 in cv::detail::FeaturesMatcher::operator()(std::__1::vector<cv::detail::ImageFeatures, std::__1::allocator<cv::detail::ImageFeatures> > const&, std::__1::vector<cv::detail::MatchesInfo, std::__1::allocator<cv::detail::MatchesInfo> >&, cv::UMat const&) ()
#16 0x0000000100491180 in cv::Stitcher::matchImages() ()
#17 0x0000000100494690 in cv::Stitcher::stitch(cv::_InputArray const&, cv::_OutputArray const&) ()
#18 0x00000001001010d0 in stitch(std::__1::vector<cv::Mat, std::__1::allocator<cv::Mat> >&) at /Users/wjs/Source/github/OpenCVSwiftStitch/SwiftStitch/stitching.cpp:82

Where is the CVWrapper.process function?

I've been searching and I can't seem to find it, where does the "CVWrapper.process" in the line
let stitchedImage:UIImage = CVWrapper.process(with: images as! [UIImage]) as UIImage
lead to?

installation using pods?

this library looks amazing !!! how can i install it using cocoa pods? the installation instructions are not 100% clear not working for me.
thanks!

UIView saved image can't stitch together but after Photoshop resave they can?

I'm working on a project that uses your library.

I tried to save the captured photos to the filesystem and later reload them when stitching. I tried several times, sometimes it worked, but most of the time it doesn't, The stitching just asking for 500MB extra memory. And then I did a test, I exported the images saved thru itunes file sharing and I resaved them AS IS using photoshop, and then I drag them back to the Images.xcassets. And then I realized, photoshop saved files work! 100% of time.

And then I tried to drag the exported photos originals back to the Images.xcassets and see if they work as well, they don't, just like the one saved in the mydocument.

I've uploaded the original photos as well as photoshop saved copies here
https://www.dropbox.com/s/2srfpi57j99e0xr/Archive.zip?dl=0

When you have time, I hope you can take a look and just drag those files to the images asset and try to stitch them. I think it must be something that I can somehow save those photos with different or some extra steps so they can be used by the stitching process

OpenCV 4 Support

This repo doesn't look that maintained anymore, but asking anyways. Does anyone have fork or version of this that supports OpenCV 4?

can not get the https://github.com/Zi0P4tch0/Specs.git

[!] Unable to add a source with url https://github.com/Zi0P4tch0/Specs.git named zi0p4tch0.
You can try adding it manually in ~/.cocoapods/repos or via pod repo add.

the error message is above. Can you help take a look? why can't I find the spec? how to fix it?

Thanks

critical issue

OpenCVSwiftStitch-master/SwiftStitch/UIImage+OpenCV.h:16:9: 'opencv2/opencv.hpp' file not found

Foundry crashes

Hi, I cloned your project and have tried running it and it worked fine. I then added my images in the extension JPG and it stitched perfectly too. So I copied the pod file, the classes and all the necessary files to my project which takes the image from the camera and saves it in gallery. I temporarily save the images taken in an array of UIImage and feed it instead of the asset images and then I get a crash in the line,
Stitcher::Status status = stitcher->stitch(imgs, pano);

I thought the problem may be in the camera images extension or orientations. But I guess it has been corrected already in the latest PR in GIT. So I copied the images that was present already in the OpenCVSwiftStitch Assets and which was working fine into my project and still I got the same crash. I could run any images in your example code and it works fine but when I do the same in mine, it fails. Is there anything that I have to add in my project to make it work?

Thanks in advance.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.