Video-rate localization in multiple maps for wearable augmented reality

Abstract

We show how a system for video-rate parallel camera tracking and 3D map-building can be readily extended to allow one or more cameras to work in several maps, separately or simultaneously. The ability to handle several thousand features per map at video-rate, and for the cameras to switch automatically between maps, allows spatially localized AR workcells to be constructed and used with very little intervention from the user of a wearable vision system. The user can explore an environment in a natural way, acquiring local maps in real-time. When revisiting those areas the camera will select the correct local map from store and continue tracking and structural acquisition, while the user views relevant AR constructs registered to that map.

Robust Global Tracking Using a Seamless Structured Pattern of Dots

“…These trackers can be subdivided into different categories. PTAM [14], PTAMM [5] and vS-LAM [12] for example build a feature map, used for tracking, at runtime. Visual Odometry [22] on the other hand only uses the information from the last few frames to determine the movement of the camera.…”

Wide-area scene mapping for mobile visual tracking

“…Most notable is the Parallel Tracking and Mapping (PTAM) system by Klein and Murray [13] and its extension to multiple maps [5] and mobile phone implementation [15]. In this approach, as the user moves a handheld camera around the scene, the 3D map is built from scratch and used for tracking and annotation.…”

“…Previous approaches typically use large-scale mapping efforts with thousands of photos [23,2], or use a real-time tracking and mapping approach which is challenging to operate in a large space [5]. Either method makes it difficult for a single user to efficiently map a large, outdoor urban area.…”

Keyframe-based real-time camera tracking

“…Recent development mainly concentrated on improving the robustness and accuracy in a larger scale scene. The major issues of this scheme include relocalisation [32,3,17,12] after camera lost, submap merging and switching [2], and the close loop management [31].…”