Head-worn mixed reality projection display application

Abstract

The main goal of this research is to develop a mixed reality (MR) application to support motion capture actors. This application allows seeing and exploring a digital environment without occluding the actor's visual field. A prototype is built by combining a retro-reflective screen covering surrounding walls and a headband consisting of a laser scanning projector with a smartphone. Built-in sensors of a smartphone provide navigation capabilities in the digital world. The integrated system has some unique advantages, which are collectively demonstrated for the first time: (i) providing fixed field-ofview (50 o in diagonal), fixed retinal images at full-resolution, and distortion-free images that are independent of the screen distance and shape; (ii) presenting different perspectives to the users as they move around or tilt their heads, (iii) allowing a focus-free and calibration-free display even on non-flat surfaces using laser scanning technology, (iv) enabling multiple users to share the same screen without crosstalk due to the use of retro-reflectors, and (v) producing high brightness pictures with a projector of only 15 lm; due to a high-gain retro-reflective screen. We demonstrated a lightweight, comfortable to wear and low cost head-mounted projection display (HMPD) which acts as a stand-a-lone mobile system. Initial informal functionality tests have been successfully performed. The prototype can also be used as a 3D stereo system using the same hardware by additionally mounting polarized glasses and an active polarization rotator, while maintaining all of the advantages listed above.

EgoSAR: Towards a Personalized Spatial Augmented Reality Experience in Multi-user Environments

“…Retroreflective displays: Retroreflective material is often used together with mobile projection systems, such as, for example, head mounted projective displays (HMPDs). Either by wearing glasses [10,8,9,7], or without glasses [16,1], this results in a personalized experience. A similar system is currently commercialized [3].…”

“…A similar system is currently commercialized [3]. However, besides [16] who raise the possibility of using perforated retroreflective material, to the best of our knowledge, previous approaches only make the augmentation on an opaque surface without taking into account the scene behind it [11,16,1].…”

Wearable Augmented Reality Displays

“…The general problems with wearable augmented reality displays (particularly near-to-eye displays) are the restricted field of view, low light efficiency and involvement of bulky optical elements. In this paper, as an alternative to near-to-eye displays, we present the wide field of view, compact and light-efficient displays based on head worn pico-projectors andretro-reflective screensthat have been recently developed in our group for 2D/3D augmented reality applications [1][2][3][4][5][6][7]. First, we introduce a compact head-worn display assembled around a regular cap.…”

“…First, we introduce a compact head-worn display assembled around a regular cap. The system includes a MEMS projector based standalone wearable display providing the required computation, connectivity and power on the head and is used together with retro-reflective surfaces as image transfer screens [1][2][3]. The light from the wearable projector is directed back with minimum scattering and provides the viewer with bright displayed content for wide range of distance.…”

Head-mounted mixed reality projection display for games production and entertainment

“…This paper is an extended version of an already published conference paper [1]. Here, we describe our prototype that we developed for a motion capture scenario and discuss how we think that this prototype could be used in other gaming, entertainment and training applications as well.…”