The idea of the camera course is to build a collision detection system - that's the overall goal for the Final Project. As a preparation for this, we have to build the feature tracking part and test various detector / descriptor combinations to see which ones perform best. This mid-term project consists of four parts:

• First, focus on loading images, setting up data structures and putting everything into a ring buffer to optimize memory load.
• Then, integrate several keypoint detectors such as HARRIS, FAST, BRISK and SIFT and compare them with regard to number of keypoints and speed.
• In the next part, focus on descriptor extraction and matching using brute force and also the FLANN approach.
• In the last part, once the code framework is complete, test the various algorithms in different combinations and compare them with regard to some performance measures.

Implemented ring buffer in which new elements are added to tail end and older items are removed from head end. It is accomplished using erase() method in vector library.

Implemented HARRIS, FAST, BRISK, ORB, AKAZE and SIFT detectors with OpenCV library and made them selectable with a string.

Removed keypoints except those on preceding vehicle by removing all keypoints detected outside a predefined reactangle (cv::Rect).

Implemented BRIEF, ORB, FREAK, AKAZE and SIFT descriptors with OpenCV library and made them selectable with a string.

Implemented FLANN matching as well as k-nearest neighbor selection and made them selectable with a string.

Implemented Descriptor Distance Ratio test which looks at the ratio of best to second-best match to decide whether to keep an associated keypoint pair. Ratio used as 0.8.

Counted the number of keypoints on the preceding vehicle for all 10 images for all the detectors implemented and analyzed their neighborhood size.

BRISK

Highest number of keypoints compared to all other detectors. Observed large and highly overlapping neighborhood size.

AKAZE

Second highest number of keypoints compared to all detectors. Some keypoints have large neihborhood size and some have small neighborhood size. Some of them are overlapped.

FAST

Higher number of keypoints compared to HARRIS, ORB, Shi Tomasi and SIFT. But lower number of keypoints compared to BRISK and AKAZE. Observed small neighborhood size.

SIFT

Higher number of keypoints compared to Shi Tomasi, ORB and HARRIS. But lower number of keypoints compared to BIRSK, AKAZE and FAST. Some keypoints have large neighborhood size and some have small neighborhood size. Few of them are overlapped.

Shi Tomasi

Higher number of keypoints compared to HARRIS and ORB. But lower number of keypoints compared to other detectors. Observed small neighborhood size.

ORB

Second least number of keypoints compared to all detectors. Some keypoints have large neighborhood size and some have small neighborhood size. Few of them are overlapped.

HARRIS

Least number of keypoints compared to other detectors and small neighborhood size.

Implemented additional logic to compute and print data for performance evaluation steps 8 and 9. Results are as below.

Comparing tradeoff between execution time and matched keypoints count, top 3 combinations are as below.

1. BRISK + BRIEF (in terms of accuracy)
2. FAST + BRIEF (in terms of speed)
3. BRISK + BRISK (average speed, best accuracy)

Note: Performance analysis results are tabulated in results/2D_FeatureMatching_Performance.csv

• cmake >= 2.8
  • All OSes: click here for installation instructions
• make >= 4.1 (Linux, Mac), 3.81 (Windows)
  • Linux: make is installed by default on most Linux distros
  • Mac: install Xcode command line tools to get make
  • Windows: Click here for installation instructions
• OpenCV >= 4.1
  • This must be compiled from source using the -D OPENCV_ENABLE_NONFREE=ON cmake flag for testing the SIFT and SURF detectors.
  • The OpenCV 4.1.0 source code can be found here
• gcc/g++ >= 5.4
  • Linux: gcc / g++ is installed by default on most Linux distros
  • Mac: same deal as make - install Xcode command line tools
  • Windows: recommend using MinGW

1. Clone this repo.
2. Make a build directory in the top level directory: mkdir build && cd build
3. Compile: cmake .. && make
4. Run it: ./2D_feature_tracking.