using OpenCV binary based feature detectors about openmvg HOT 12 CLOSED

Allow few comments,

• First, OpenCV (with all respect to community) lacks of "good" implemantation of BRISK, FREAK descriptors. They are present and working there but simply under perform, several bugs are already filled regarding the issue. I suggest to use original brisk and freak (BSD labeled) code along original agast++ detector code. ORB, FAST can be used out of OpenCV's original, those are in excelent shape, however they are poourish detector/descriptors. Same story goes with SIFT, i recomend to use vlfeat of A. Vedaldi. I think OpenCV can be avoided at all if we drop ORB+FAST.

  • Also, we can look at BinBoost: http://cvlab.epfl.ch/research/detect/binboost that is consideret state-of-art and outperforms SIFT.

  FLANN can be used with HierarhicalClustering() instead of LSH, LSH requires huge amount of vectors to be preloaded for beeing efficient, HS performs much better in terms of precision and low amount of vectors. Also, for some reason HS underferform in 1.8x but works better in 1.7.x (need to fill bug report), but 1.7.x require TBB instead of OpenMP for parallelism. FLANN's internal L2/Hamming functor can be extended/overwritten externaly with nice SSE3.1/SSE4.2 inlines for a 10% speedup.

  As for bruteforce matching schema, i should use OpenCV internal, its more than easy to write our own ones (CUDA variant too) for booth hamming/euclid, even in bare C. OpenCV requires special data structures for holding keypairs.

• I try today push a sample tool "feat2d" written by myself to evaluate what i described here.

from openmvg.

Can be done by using the actual descriptor class and updating the metric to do the appropriate hamming distance.

from openmvg.

Could you provide an example where to update this metric and what value to use?

from openmvg.

It looks like this is where one could implement a new Hamming distance metric:

https://github.com/openMVG/openMVG/blob/7510e8733488c90a8da5d4a474f1ef779a993f5b/src/openMVG/matching/metric.hpp

from openmvg.

Hamming distance metric could be implemented with the ^ operator to compute common 1 bits and after you have to sum the 1.

mySum += popcount(val1 ^ val2);

You will find information about population counting:
See http://wm.ite.pl/articles/sse-popcount.html
http://chessprogramming.wikispaces.com/Population+Count
http://msdn.microsoft.com/fr-fr/library/bb385231(v=vs.90).aspx
https://github.com/hanji/popcnt/blob/master/populationcount.cpp

from openmvg.

I just implemented this in JavaScript for use with BRISK. Perhaps in the
future I can contribute the Hamming metric and updates to the pipeline to
support the selection of different metrics.
On Feb 12, 2014 2:00 PM, "Pierre Moulon" [email protected] wrote:

Hemming distance metric could be implemented with the ^ operator to
compute common 1 bits and after you have to sum the 1.

mySum += popcount(val1 ^ val2);

You will find information about population counting:
See http://wm.ite.pl/articles/sse-popcount.html
http://chessprogramming.wikispaces.com/Population+Count
http://msdn.microsoft.com/fr-fr/library/bb385231(v=vs.90).aspx
https://github.com/hanji/popcnt/blob/master/populationcount.cpp

Reply to this email directly or view it on GitHubhttps://github.com//issues/32#issuecomment-34923288
.

from openmvg.

Dear all,

Just a comment regarding the Hamming distance. It is already implemented in FLANN which is used as a third-party in openMVG. The implementation is already strong and rather easy to understand IMHO.
To convince oneself, see lines 368-578 of file: https://github.com/openMVG/openMVG/blob/master/src/third_party/flann/src/cpp/flann/algorithms/dist.h

Therefore, we can match binary descriptors

• either by directly using flann::LSHIndex
• or by customizing the template class openMVG::matching::ArrayMatcherBruteForce (if you don't feel like trusting LSHIndex) as follows:

typedef typename openMVG::matching::Hamming<unsigned char> HammingDistance;
typedef typename openMVG::matching::ArrayMatcherBruteForce<unsigned char, HammingDistance> BinaryBruteForceMatcher;

where I assume that the template class:

• template < typename T > struct openMVG::matching::Hamming< T > would be written in the file 'https://github.com/openMVG/openMVG/blob/master/src/openMVG/matching/metric.hpp'
• The template functor would be a mere wrapper of the existing functor template < typename T > struct flann::Hamming< T > as defined in: https://github.com/openMVG/openMVG/blob/master/src/third_party/flann/src/cpp/flann/algorithms/dist.h

HTH.

from openmvg.

Yep I agree with @davidok8. There is also an implementation of Hierarchical Clustering Index in Flann.

from openmvg.

Errata: "SSSE3/SSE4.2", SSE3.1 does not exist, its a typo.

from openmvg.

http://openrisc.rdsor.ro/featsfm/

There is a tarball containing:

• feat2d (descriptor/matcher) with all descriptors possible.
• bundler (with SBA+PBA , and HDF5 i/o).
• pmvs2 (with minor sse optimization).
• Dependencies are quite big, i use to pack rpm packages for fedora so follow these externals:
• need OpenCV with non_free module
• need to have all libs from this URL:
  https://code.google.com/p/featsfm/source/browse/?r=26#svn%2Ftrunk%2Ffedora%2Fsrc
• If not familiar with redhat/fedora rpm packages you can unpack them , follow .spec file and patch orders.

from openmvg.

Hi there,

This request will be considered as a feature for the 0.6 milestone:
#88

But as said @davidok8 it can already used easily:
Change SIFT template descriptor to a binary one:

// Sift descriptor
// typedef Descriptor < unsigned char, 128 > DescriptorT;
// Binary one: 512 bits:
typedef Descriptor < unsigned char, 64 >  DescriptorT;
// Or using existing STL datastructure
typedef Descriptor < std::bitset< 512 >, 1 > DescriptorT;

After you have just to use the associated good metric:

• With raw memory:
  • flann Hamming metric:

  typedef flann::Hamming<DescriptorT::bin_type> MetricT;

• With bitset (add the metric in metric.hpp) (enable SSE4 support to be fast on Unix):
  • src/CmakeLists.txt

IF (UNIX)
    SET(CMAKE_C_FLAGS_RELEASE "-msse2 -msse3 -msse4")
    SET(CMAKE_CXX_FLAGS_RELEASE "-msse2 -msse3 -msse4")
ENDIF (UNIX)

• openMVG/Matching/metric.hpp

template< typename TBitset>
struct HammingBitSet
{
  typedef TBitset ElementType;
  typedef int ResultType;

  // Returns the Hamming Distance between two dbrief descriptors
  template <typename Iterator1, typename Iterator2>
  ResultType operator()(Iterator1 a, Iterator2 b, size_t size) const
  {
    return (*a ^ *b).count();
  }
};

Sample usage:

  Image<uchar> I; // Suppose to contain a valid iamge
  cv::Mat img;
  cv::eigen2cv(I.GetMat(), img);

  std::vector< cv::KeyPoint > vec_keypoints;
  cv::Mat m_desc;

  cv::SurfFeatureDetector detector(2000,4);
  // sample with FREAK (512 bits)
  cv::FREAK extractor;

  detector.detect( img, vec_keypoints );
  extractor.compute( img, vec_keypoints, m_desc );

  // convert feature and descriptor to openMVG internal format see main_computeMatches-OpenCV.cpp

  // Define metric and matcher
  typedef flann::Hamming<DescriptorT::bin_type> MetricT;
  typedef ArrayMatcherBruteForce<DescriptorT::bin_type, MetricT> MatcherT;

 // Match => set the Distance ratio higher than 0.6, since that binary descriptor are mode sensible to this criteria. I.E use -r 0.9 option

from openmvg.

I'd be really interested in hearing anyone's experience trying these binary descriptors. Specifically how much speedup they get vs. matching accuracy.

from openmvg.