Real-Time Detection and Tracking for Augmented Reality on Mobile Phones

Abstract

In this paper, we present three techniques for 6DOF natural feature tracking in real time on mobile phones. We achieve interactive frame rates of up to 30 Hz for natural feature tracking from textured planar targets on current generation phones. We use an approach based on heavily modified state-of-the-art feature descriptors, namely SIFT and Ferns plus a template-matching-based tracker. While SIFT is known to be a strong, but computationally expensive feature descriptor, Ferns classification is fast, but requires large amounts of memory. This renders both original designs unsuitable for mobile phones. We give detailed descriptions on how we modified both approaches to make them suitable for mobile phones. The template-based tracker further increases the performance and robustness of the SIFT- and Ferns-based approaches. We present evaluations on robustness and performance and discuss their appropriateness for Augmented Reality applications.

Using multiple sensors for reliable markerless identification through supervised learning

“…The adoption of diffused illumination would normally allow the use of fiducial markers to perform recognition. For this reason our first prototype was based on ARToolkit+ [34] (see Fig. 6), a widely used extension to the well-known ARToolkit tag system [22].…”

Unified Real-Time Tracking and Recognition with Rotation-Invariant Fast Features

“…With the introduction of highly capable smartphones, there has been much recent interest in combining tracking and content recognition for MAR [2,3,4,5,6,1]. Systems have been proposed to track video content using feature descriptors, including motion vector tracking [7], SURFTrac [2], and patch tracking [5].…”

“…Wagner et al [4,5,6] have made significant progress towards such systems on mobile phones. However, their systems use different methods for tracking (PatchTracker) and for recognition (PhonySIFT or PhonyFERNS).…”

A 345 mW Heterogeneous Many-Core Processor With an Intelligent Inference Engine for Robust Object Recognition

“…However, this reduction would be meaningless if the selected tiles do not contain the target object. In tile-based parallel object recognition, the attention precision can be defined as (1) Using this definition of attention precision, the recognition effectiveness is defined as (2)…”

“…OBUST object recognition is the key component in vision-based applications such as augmented reality [1], content-based image retrieval, and intelligent robots. In these applications, local descriptor matching object recognition algorithms, such as the Scale Invariant Feature Transform (SIFT) [2], are widely used due to their invariance to scaling, rotation, and illumination.…”