Video-rate recognition and localization for wearable cameras

Abstract

Using simultaneous localization and mapping to determine the 3D surroundings and pose of a wearable or hand-held camera provides the geometrical foundation for several capabilities of value to an autonomous wearable vision system. The one explored here is the ability to incorporate recognized objects into the map of the surroundings and refer to them. Established methods for feature cluster recognition are used to identify and localize known planar objects, and their geometry is incorporated into the map of the surrounds using a minimalist representation. Continued measurement of these mapped objects improves both the accuracy of estimated maps and the robustness of the tracking system. In the context of wearable (or hand-held) vision, the system's ability to enhance generated maps with known objects increases the map's value to human operators, and also enables meaningful automatic annotation of the user's surroundings.

Video-rate localization in multiple maps for wearable augmented reality

“…Integration of the object recognition work done previously [2], will allow a richer AR experience around particular real world objects for the user. It may also allow semantic links between overlapping maps that contain the same objects.…”

“…Three different experiments are presented in the paper: (1) the "desk" sequence shows the robustness, and some limitations of the system, and how the multiple mapping works; (2) the "laboratory" sequence shows how the system can use overlapping maps, and handle mobile objects; (3) the "building" sequence shows how the system can be used to explore large, sprawling environments. The experiments were performed using two cameras, the hand-held camera and the shoulder mounted active camera described above.…”

“…It was demonstrated operating within the Extended Kalman Filter (EKF) monoSLAM algorithm of Davison et al [3,4], a method which has already proved a useful vehicle for AR using wearable vision [16,2].…”

Visual simultaneous localisation and map-building supported by structured landmarks

“…In (Castle et al, 2007a ;2007b), some known planar patterns were used together with point-features. The planar pattern recognition sequence runs on the PC second core as a separate thread at around 1.5 Hz rate.…”

A CNN-SIFT Hybrid Pedestrian Navigation Method Based on First-Person Vision

“…First-person vision (FPV) [2] is arguably the most commonly used, but other names, such as egocentric vision and ego-vision, have also recently grown in popularity. There are many objectives of FPV analysis, such as object recognition and tracking [3][4][5][6], activity recognition [7][8][9][10][11], and environment mapping [12][13][14][15].…”