Notes on Feature Detector and Descriptor.
It is not so hard to use the APIs of feature detector and descriptors, but not so trivial to find the right/possible usage of them. And a nice workflow may be appreciate.
- Detect the feature points and descriptors
- SIFT, SURF / ORB, BRIEF, BRISK and others
- Match with the distance of descriptors
- NORM_L2, NORM_HAMMING
- Refine the matches
- correctMatches, with Homography, Fundmental
- Calculate the R,t
- decomposeEssentialMat/recoverPose, decomposeHomographyMat
OpenCV 3: List of available FeatureDetector::create() and DescriptorExtractor::create() options?
Other solution is to test each feature:
if the call to detect() is ok (no exception thrown) ==> feature detection if the call to compute() is ok ==> feature extraction if the call to detectAndCompute() is ok ==> both or looking directly into the source code.
BRISK: detector + descriptor
ORB: detector + descriptor
MSER: detector
FAST: detector
AGAST: detector
GFFT: detector
SimpleBlobDetector: detector
KAZE: detector + descriptor
AKAZE: detector + descriptor
FREAK: descriptor
StarDetector: detector
BriefDescriptorExtractor: descriptor
LUCID: descriptor
LATCH: descriptor
DAISY: descriptor
MSDDetector: detector
SIFT: detector + descriptor
SURF: detector + descriptor
The default type of descriptors and distance calculate method (norm):
// To know which norm type to use: OpenCV 3.x, OpenCV 2.x do not have the `NormTypes` enum.
Ptr<Feature2D> akaze = AKAZE::create ();
std::cout << "AKAZE: " << akaze->descriptorType() << " ; CV_8U=" << CV_8U << std::endl;
std::cout << "AKAZE: " << akaze->defaultNorm() << " ; NORM_HAMMING=" << cv::NORM_HAMMING << std::endl;cv2.NORM_L2(default): good for SIFT, SURF
cv2.NORM_HAMMING: ORB, BRIEF, BRISK
cv2.NORM_HAMMING2: ORB with VTA_K == 3 or 4
There is another prove out there: How Does OpenCV ORB Feature Detector Work?
It is okay to accept if things are so.
I do believe the performance of pose estimation really relays on how good the matches are, so the refine part is more than crucial.
-
- Use H and F
-
undistortPoints, findEssentialMat, recoverPose: What is the relation between their arguments?
- Use
correctMatches
- Use
Parameter used for RANSAC. It is the maximum distance from a point to an epipolar line in pixels, beyond which the point is considered an outlier and is not used for computing the final fundamental matrix. It can be set to something like 1-3, depending on the accuracy of the point localization, image resolution, and the image noise.
So, the defaults value of 3.0f can be a little big.
crossCheck which is false by default. If it is true, Matcher returns only those matches with value (i,j) such that i-th descriptor in set A has j-th descriptor in set B as the best match and vice-versa. That is, the two features in both sets should match each other. It provides consistant result, and is a good alternative to ratio test proposed by D.Lowe in SIFT paper.
So, it won't hurt to do this check, right?
- ORB
Feature detector and descriptor for low resolution images
The nfeatures, the default 500 can be too small for many cases.
scaleFactor: Pyramid decimation ratio, maybe we can use larger values for larger images?
# OpenCV 3.x
orb = cv2.ORB_create(nfeatures=5000)