1. Clone this repo.
2. Make a build directory in the top level directory: mkdir build && cd build
3. Compile: cmake .. && make
4. Run it: ./2D_feature_tracking.

Steps to install are taken from OpenCV's official instructions, but since we are going to use some algorithms from xfeatures2d.hpp and xfeatures2d/nonfree.hpp an additional flags need to be set.

# Download and unpack sources
wget -O opencv.zip https://github.com/opencv/opencv/archive/master.zip
wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/master.zip
unzip opencv.zip
unzip opencv_contrib.zip

# Create build directory and switch into it
mkdir -p build && cd build

# Configure, need to set flag for non-free, and contrib (extra module)
# --Note-- Check note below.
cmake -DOPENCV_ENABLE_NONFREE=ON -DOPENCV_EXTRA_MODULES_PATH=../opencv_contrib-master/modules ../opencv-master

# Build (this step will take quite long)
cmake --build .
make -j8

# Install
sudo make install

In configure step, make sure to check an output message. It should show something similar to the following lines.

<some other messages>
...
-- Configuring done
-- Generating done
-- Build files have been written to: /home/xxx/Documents/Udacity/build

If not, you problably missing some required libraries. Read through the output messages to figure out what's missing, then delete and create another build directory and try to run the configure command again.

After install OpenCV successfully, you will have the following directory installed on your system (with default install path).

• /usr/local/bin - executable files
• /usr/local/lib - libraries (.so)
• /usr/local/cmake/opencv4 - cmake packages
• /usr/local/include/opencv4 - headers
• /usr/local/share/opencv4 - other files

In Visual Studio Code:

• Press Ctrl+Shift+p to bring up the drop-down menu.
• Select C/C++: Edit Configurations (UI)
• In C/C++ Configurations tab, in Include path section, put in OpenCV's header directory's path.
• Put path to header file directory as /usr/local/include/opencv4/**, the preceeding /** tell the IDE to search for header files within the subfolders as well.

Dropdown menu pop-up after press Ctrl+Shift+p

In Include path section, add /usr/local/include/opencv4/**

When done, you should be able to see a command list

command pallete

When consider a result from difference detector applied on the same set of images, we can see that ORB detector provide the highest ratio of keypoints within our focus area versus the whole frame (around 20%), and also taken shortest amount of time required (except only the first frame).

Note : Keypoints are dense and distributed across the frame.

Shi-Tomasi Detector

Note : Provide very few keypoints compare to other methods.

Harris Detector

Note : Provide dense keypoints with different sizes and lots of overlapping. Many keypoints are on feature like overhead bridge and trees, which is irrelevant to us.

BRISK Detector

Note : Majority of keypoints are overlapping and have about the same size. Good amount of keypoints are in our focus area (more than 20%).

ORB Detector

Note : Keypoints are distribute evenly across the frame, not many of them overlapping. Most of the keypoints have the same size.

AKAZE Detector

Note : Keypoint are of the difference size, few of them are overlapping.

SIFT Detector

Number of matched keypoints and descriptor extraction time from various Detector and Descriptor pair are listed below.

HARRIS Detector with BRIEF, ORB, FREAK and SIFT Descriptor

FAST Detector with BRIEF, ORB, FREAK and SIFT Descriptor

BRISK Detector with BRIEF, ORB, FREAK and SIFT Descriptor

ORB Detector with BRIEF, ORB, FREAK and SIFT Descriptor

AKAZE Detector with AKAZE Descriptor

SIFT Detector with BRIEF, FREAK and SIFT Descriptor

By taking an average number of macthed descriptor and extraction time of all 10 frames, we can conclude the data together. As the data shown, combination of BRISK Detector with BRIEF Descriptor yield the best average keypoints matched and description extraction time.

The Top-3 Detector&Descriptor, judge from both number of keypoints and extarction time is then:

• 1st BRISK Detector with BRIEF Descriptor
• 2nd BRISK Detector with ORB Descriptor
• 3rd BRISK Detector with SIFT Descriptor

BRISK Detector with BRIEF Descriptor yield the best average keypoints matched and description extraction time.

From above table, BRISK Detector yeild highest number of keypoints matched, which we can infer that the keypoints and BRISK detect has both quantity and resilient cross frames. In term of Descriptor, both BRIEF and ORB took shortest time for descriptor extraction compared to another three.