Enhancing the Point Feature Tracker by Adaptive Modelling of the Feature Support

Abstract

Abstract. We consider the problem of tracking a given set of point features over large sequences of image frames. A classic procedure for monitoring the tracking quality consists in requiring that the current features nicely warp towards their reference appearances. The procedure recommends focusing on features projected from planar 3D patches (planar features), by enforcing a conservative threshold on the residual of the difference between the warped current feature and the reference. However, in some important contexts, there are many features for which the planarity assumption is only partially satisfied, while the true planar features are not so abundant. This is especially true when the motion of the camera is mainly translational and parallel to the optical axis (such as when driving a car along straight sections of the road), which induces a permanent increase of the apparent feature size. Tracking features containing occluding boundaries then becomes an interesting goal, for which we propose a multi-scale monitoring solution striving to maximize the lifetime of the feature, while also detecting the tracking failures. The devised technique infers the parts of the reference which are not projected from the same 3D surface as the patch which has been consistently tracked until the present moment. The experiments on real sequences taken from cars driving through urban environments show that the technique is effective in increasing the average feature lifetimes, especially in sequences with occlusions and large photometric variations.The presented work has been performed within the french national project Predit Mobivip (Individual Public Vehicles for Mobility in the City), and within the project Robea Bodega. The authors would also like to acknowledge Prof. Axel Pinz for a helpful discussion on this subject.

Histogram-Based Description of Local Space-Time Appearance

“…The descriptor is calculated in every frame using the current frame information and the information from previous frames. Tracking can be achieved either by detection, or by using a standard tracker such as meanshift or KLT [17]. The algorithm for descriptor calculation assumes that a bounding box around the object of interest is available in every frame.…”

Large scale vision-based navigation without an accurate global reconstruction

“…Both components rely on a multi-scale differential tracker with warp correction and checking towards the reference appearance [19]. The employed warp includes isotropic scaling and affine contrast compensation [18]. The output of the framework is a set of 2D vectors connecting the current features with their corresponding locations in the next key-image.…”

“…The above is similar to visual odometry [14], except that we employ larger feature windows and more involved tracking in order to achieve more distinctive features and longer feature lifetimes. To ensure a minimum number of features within an arc of the graph, a new node is forced when the absolute number of tracked points falls below n. Bad tracks are identified by a threshold R on RMS residual between the current feature and the reference appearance [19,18]. Typically, the following values were used: σ = 4, n = 50, R = 6.…”

A Framework for Scalable Vision-Only Navigation

“…Bad tracks are identified by a threshold R on RMS residual between the warped current feature and the reference appearance. The employed warp includes isotropic scaling and affine contrast compensation [18]. The two-view geometry is recovered in a calibrated context by random sampling, with the five-point algorithm [19] as the hypothesis generator.…”

“…Then the current frame is discarded, while the previous frame is registered as the new node of the graph, and the whole procedure is repeated from there. This is similar to visual odometry [20], except that we employ larger feature windows and more involved tracking [18] in order to achieve more distinctive features and longer feature lifetimes. To ensure a minimum number of features within an arc of the graph, a new node is forced when the absolute number of tracked points falls below n.…”