An occlusion capable optical see-through head mount display for supporting co-located collaboration

Abstract

An ideal augmented reality (AR) display for multi-user co-located collaboration should have following three features: 1) Any virtual object should be able to be shown at any arbitrary position, e.g. a user can see a virtual object in front of other users' faces. 2) Correct occlusion of virtual and real objects should be supported.3) The real world should be naturally and clearly visible, which is important for face-to-face conversation. We have been developing an optical see-through display, ELMO (Enhanced see-through display using an LCD panel for Mutual Occlusion), that satisfies these three requirements. While previous prototype systems were not practical due to their size and weight, we have come up with an improved optics design which has reduced size and is lightweight enough to wear. In this paper, the characteristics of typical multi-user three-dimensional displays are summarized and the design details of the latest optics are then described. Finally, a collaborative AR application employing the new display and its user experience are explained.

A survey of diminished reality: Techniques for visually concealing, eliminating, and seeing through real objects

“…Optical see-through (OST) systems in DR have been lagging the technical difficulties of "shutting off light rays from the real object" that is the preconditioning procedure of DR. In other words, we also expect the emergence of a reasonable solution for occlusion problems of OST display in AR/MR [79].…”

Color correction for optical see-through displays using display color profiles

“…Solving the occlusion problem keeps digital content from being affected by the physical objects in the background, thus solving color blending. The main approach to occlusion has been to stop the light coming from the background by enhancing head-mounted displays with spatial light modulation (SLM) devices [Cakmakci et al 2004] [Kiyokawa et al 2003][2002] [Zhou et al 2007]. In this approach a black/white depth mask of the scene is generated with the black pixels covering the area where digital content is not to mix with the background light.…”

“…The degree of display opacity poses a challenge for how much accessibility and clarity is needed for either the background or the display content, and remains an open question. Ideally, an optical see-through display should provide a way to control the level of transparency at a pixel level, as explored by [Kiyokawa et al 2003]. A rendering pipeline could rely on BP-based color correction for darker backgrounds and block background light as the correction algorithm approaches its limits (i.e., when high intensity background colors makes the display color uncorrectable).…”

“…Color blending is an important issue as it affects the legibility and colorencodings of digital information and compromises the general usability of such devices. Existing solutions include using a spatial light modulator (SLM) to block background light [Kiyokawa et al 2002[Kiyokawa et al ][2003, an approach requiring extra hardware on the display at the cost of non-transparency. Color correction is another solution where the system finds an alternative digital color which, upon blending with the background, comes closest to the desired color [Weiland et al 2009].…”

Simulation-based design and rapid prototyping of a parallax-free, orthoscopic video see-through head-mounted display

“…[4]), VST-HMDs offer unique advantages [5]. Our group has been developing guidance systems with VST-HMDs for minimally invasive surgery.…”