An empirical study of non-rigid surface feature matching

Abstract

This paper presents an empirical study of affine invariant feature detectors to perform matching on video sequences of people with non-rigid surface deformation. Recent advances in feature detection and wide baseline matching have focused on static scenes. Video frames of human movement captures highly non-rigid deformation such as loose hair, cloth creases, skin stretching and free flowing clothing. This study evaluates the performance of three widely used feature detectors for sparse temporal correspondence on single view and multiple view video sequences. Quantitative evaluation is performed of both the number of features detected and their temporal matching against and without ground truth correspondences. Recall-accuracy analysis of feature matching is reported for temporal correspondence on single view and multiple view sequences of people with variation in clothing and movement. This analysis identifies that existing feature detection and matching algorithms are unreliable for fast movement with common clothing. For patterned clothing techniques such as SIFT produce reliable correspondence.

Multi-Camera Active-Vision for Markerless Shape Recovery of Unknown Deforming Objects

“…One example method in [55] is capable of markerless shape and motion capture from multi-view video sequences, given complete visibility of the actor. Passive, multi-camera feature-based methods are, typically, less susceptible to lighting conditions, and do not require object manipulation [17], [56].…”

“…Common feature descriptors include: scale-invariant feature transform (SIFT) [83], [84], speeded up robust features [85], gradient location and orientation histogram (GLOH) [86], and histogram of Gaussians [1]. Most of these methods have led to derivative methods such as ASIFT [87], HesAff-GLOH, HesAff-Sift, and MSER-SIFT [56]. It should be noted that even the GLOH method is an extension of SIFT with the integration of PCA to further reduce the 128-integer key produced by SIFT down to 64-integers [88].…”

“…Conversely, robust, scale-invariant features such as scale-invariant feature transform [83], [84] (SIFT) or speeded-up robust features [85] (SURF), or alternatives [56], allow for larger baseline and angular separation due to their feature matching robustness under affine distortions [84]. The resulting list of stereocamera pairs is stored in KTR, a [2 × kT] matrix, where kT is the number of viable pairs.…”

“…They are, also, typically more robust to the multi-camera baseline length [84]. Most notable examples of such methods have been thoroughly tested for robustness in recall rates [56].…”

Self-calibration of Large Scale Camera Networks

“…find the matches from image 1 to image 2, and cross-check with the matches from image 2 to image 1. A number of pairwise feature detectors were tested (Doshi et al, 2010), including the SIFT (Lowe, 2004) and SURF (Bay et al, 2006), where SIFT proved to provide the most reliable matches on our dataset. The configuration of the cameras determine the exact approach for finding matches.…”