VulkanSift is a GPU implementation of the Scale-Invariant Feature Transform algorithm by David G. Lowe. The library is written in C and uses Vulkan to run SIFT feature detection and matching algorithms on any GPU supporting the Vulkan API.

Main features:

• SIFT feature detection and matching (2 nearest-neighbors brute-force matching)
• Configurable SIFT implementation (upsampling option, scalespace dimension, kernels sigma, extremum detection thresholds)
• Multiple descriptors format supported for compatibility with descriptors from other implementations
• Non-blocking API for time-consuming functions (detect/match)

Detection speed and feature comparison with other implementations are detailed in the Performances page.

Note (31/03/21): as of today, VulkanSIFT has been tested with the following GPU/platforms:

• NVIDIA RTX 2060 (laptop): Linux/Windows
• NVIDIA GTX 960M (laptop): Linux
• Qualcomm Adreno 615 (smartphone): Android

By default, VulkanSift doesn't have any dependency and load the Vulkan library at runtime. If the Vulkan requirements are not met at runtime, the library setup calls will gracefully fail to let users fall back to other SIFT implementations.

The library examples use OpenCV to read image files and display results.

(VulkanSift development only) GLFW3 is used in an optional test program to provide an interface to any supported OS window manager. This is required to allow VulkanSift calls to be debugged/profiled with GPU tools (NVIDIA Nsight, RenderDoc, etc).

Optionnal performance test programs need to link and include files from other SIFT implementations (VLFeat, SiftGPU, PopSift, OpenCV). Information on how to add the required files to this repository are present in Performances.

• CMake (version >= 3.0)
• Python3
• Vulkan SDK

Clone the github repository and its submodules:

git clone --recurse-submodules [email protected]:maelaubert/VulkanSift.git

To build in a build/ folder in the repository folder:

mkdir build && cd build
cmake ..
make 

This CMake project provides the following options that you can use with your cmake command:

To install the CMake project (binaries, library file and include files) in the folder defined by the CMAKE_INSTALL_PREFIX variable:

# depending on CMAKE_INSTALL_PREFIX admin rights may be needed
make install

If VulkanSift is used from another CMake project and has been installed you can use the CMake command:

find_package(VulkanSift)
target_link_libraries(TARGET ${VulkanSift_LIB})
target_include_directories(TARGET PRIVATE ${VulkanSift_INCLUDE_DIR})

If not found automatically (may happen with custom install location) the CMake option VulkanSift_DIR can be set to specify the path to the folder containing VulkanSiftConfig.cmake.

If VulkanSift is not installed, you can place VulkanSift direcly inside your CMake build tree and use:

add_subdirectory(PATH_TO_VULKANSIFT/VulkanSift)
target_link_libraries(TARGET ${VulkanSift_LIB})
target_include_directories(TARGET PRIVATE ${VulkanSift_INCLUDE_DIR})

#include <vulkansift/vulkansift.h>

int main()
{
  // Load and get acces to the Vulkan API
  if (vksift_loadVulkan() != VKSIFT_SUCCESS)
  {
    // Vulkan library not found, there may be no device supporting Vulkan
    // on this machine or something failed when setting up the Vulkan API access.
    return -1;
  }

  // Create VulkanSift instance using the default configuration (automatically selected GPU)
  vksift_Config config = vksift_getDefaultConfig();
  vksift_Instance vksift_instance = nullptr;
  if (vksift_createInstance(&vksift_instance, &config) != VKSIFT_SUCCESS)
  {
    // May fail if selected GPU doesn't support the application requirement or if something
    // went wrong when setting up/preparing GPU ressource memory or detection/matching pipelines.
    return -1;
  }

  uint8_t *image_data;
  uint32_t image_width;
  uint32_t image_height;
  // Fill above variables with image data/information here...

  // Start feature detection on the image, store the detection results in the GPU buffer 0
  uint32_t gpu_buffer_idx = 0u;
  vksift_detectFeatures(vksift_instance, image_data, image_width, image_height, gpu_buffer_idx);
  // Setup and resize a std::vector to store exactly the number of detected features
  std::vector<vksift_Feature> sift_features;
  sift_features.resize(vksift_getFeaturesNumber(vksift_instance, gpu_buffer_idx));
  // Download the detection results into the std::vector buffer
  vksift_downloadFeatures(vksift_instance, sift_features.data(), gpu_buffer_idx);

  // Destroy VulkanSift instance and release Vulkan API
  vksift_destroyInstance(&vksift_instance);
  vksift_unloadVulkan();

  return 0;
}

The following examples are available in the src/examples folder:

• test_sift_detect: detect and display SIFT features.
• test_sift_match: detect features from two images on two different GPU buffer, perform brute-force feature descriptor matching between the two buffer, download and filter the results on the CPU (cross-check, Lowe's ratio check) and display the valid matches.
• test_sift_show_pyr: detect SIFT feature on the input image, then display interactive visualization of the pyramid images downloaded from the GPU (scale-space images, Difference of Gaussian images).
• test_sift_error_handling: shows how to configure a vksift_Instance to use a user-provided error callback function that throws an exception on errors. This code intentionally causes an invalid input error and shows that exceptions are used.