Information Overlay for Camera Phones in Indoor Environments

Abstract

Abstract. Increasingly, cell phones are used to browse for information while location systems assist in gathering information that is most appropriate to the user's current location. We seek to take this one step further and actually overlay information on to the physical world using the cell phone's camera and thereby minimize a user's cognitive effort. This "magic lens" approach has many applications of which we are exploring two: indoor building navigation and dynamic directory assistance. In essence, we match "landmarks" identified in the camera image with those stored in a building database. We use two different types of featuresfloor corners that can be matched against a floorplan and SIFT features that can be matched to a database constructed from other images. The camera's pose can be determined exactly from a match and information can be properly aligned so that it can overlay directly onto the phone's image display. In this paper, we present early results that demonstrate it is possible to realize this capability for a variety of indoor environments. Latency is shown to already be reasonable and likely to be improved by further optimizations. Our goal is to further explore the computational tradeoff between the server and phone client so as to achieve an acceptable latency of a few seconds.

A review of assistive spatial orientation and navigation technologies for the visually impaired

“…The position of the pedestrian was provided when a match was found between an image taken by the pedestrian and an image stored in the database [52]. The database and the image processing could be made locally or remotely on a server, depending on processing power requirements [53]. A visual pedestrian navigation system independent of a server and of pre-existing databases usually needs integration with other positioning sensors to be functional.…”

Landmark-Based Pedestrian Navigation with Enhanced Spatial Reasoning

“…-We support automatic live augmentation of images taken by the user, which was deemed important in the user feedback on our previous system. Instead of using a system based on matching to a 3-D reconstruction as is done in Photo Tourism and other previous work [13,14], we propose a new method based on image-to-image matching that can work with a system designed to run in real-time on a camera phone [6]. -Additionally, since our technique automatically aligns the 3D reconstruction of a landmark with the world data, the system can be further extended to support other features that users asked for in our initial study: features such as zooming out to give users additional context information (for example street information), warping images so that they appear as if they were taken from the current location, highlighting a portion of an image that contains the landmark, etc.…”

Metaphor and Manifestation Cross-Reality with Ubiquitous Sensor/Actuator Networks

“…Evolving approaches using coarse radio or sporadic GPS location to constrain a computer vision system hint at a tractable, robust indoor location capability that could improve the close-range performance of such systems. 14 The emergence of 3D augmented reality on smart phones 15 that can query sensor networks and connect with shared online worlds promises to gestate immersive mobile cross-reality.…”