Multiple types of keypoints and descriptors about kapture HOT 6 CLOSED

Hi Dmytro!

Thanks for your interest in this. Indeed, we already discussed a lot how to support multiple types of local features. We will continue the discussion and get back to you as soon as possible.

Best,
Martin

from kapture.

Thank you, Martin! If possible, I would be happy to join the discussion :)
My email is [email protected] just in case

from kapture.

Great! It would also be helpful if you could share some more details about your needs. At the moment we use kapture with multiple features, but not within the same reconstruction.
Best, Martin

from kapture.

Sure. The general idea is to revisit MODS, which combines the multiple type of features in order to speed-up matching of the easy pairs and to allow matching the hard pairs, where any single feature fails, but combined they work.
So, first we are matching everything which is matchable by ORB, then we add, say, SuperPoint, and so on.

https://github.com/ducha-aiki/mods-light-zmq
https://arxiv.org/abs/1503.02619

from kapture.

Hi,
We've started to draft a new version of the format (1.1) that should hopefully answer your needs.
The new specifications are being written here
https://github.com/naver/kapture/blob/1.1_draft/kapture_format.adoc

There are no changes to the "sensor" part of the format.
In the "reconstruction" part, multiple keypoints/descriptors/global features types can be given. keypoints can be represented by multiple descriptors.
Observations can reference different types of keypoints.

What do you think of these changes ?

from kapture.

Hi,

It fits my use-case, thank you!
One thing, I would like to add is a metric for descriptors. So the descriptors.txt would be:

name, dtype, dsize, keypoints_type, metric_type

where the metric is "L2" for the most and "Hamming" for the binary features, for example.

from kapture.