Head-mounted mixed reality projection display for games production and entertainment

Abstract

This research presents a mixed reality (MR) application that is designed to be usable during a motion capture shoot and supports actors with their task to perform. Through our application, we allow seeing and exploring a digital environment without occluding an actor's field of vision. A prototype was built by combining a retroreflective screen covering surrounding walls and a headband consisting of a laser scanning projector with a smartphone. Built-in sensors of a smartphone provide navigation capabilities in the digital world. The presented system was demonstrated in an initially published paper. Here, we extend these research results with our advances and discuss the potential use of our prototype in gaming and entertainment applications. To explore this potential use case, we built a gaming application using our MR prototype and tested it with 45 participants. In these tests, we use head movements as rather unconventional game controls. According to the performed user tests and their feedback, our prototype shows a potential to be used for gaming applications as well. Therefore, our MR prototype could become of special interest because the prototype is lightweight, allows for freedom of movement and is a lowcost, stand-alone mobile system. Moreover, the prototype also allows for 3D vision by mounting additional hardware.

A Large Effective Touchscreen Using a Head-Mounted Projector

“…The images are projected from a projector placed at a position that is optically conjugate with the observer's eyes, and the projected images are reflected on an object on which a retroreflective material is applied and reaches the eyes through the half mirror. Also, there are systems in which images from a projector are directly projected on a flat surface on which a retroreflective material is applied, without using a half mirror [11,12].…”

“…On the other hand, there have been studies on UIs using a head-mounted projector that can constantly display images in front of the user [8][9][10][11][12][13][14][15]. In these studies, it is possible to present images in his or her gaze direction, since the projector is worn on the head.…”

Transdisciplinarity Seen Through Information, Communication, Computation, (Inter-)Action and Cognition

“…A challenge for interdisciplinary teams can, as we experienced in our project (Kade 2014;Kade et al 2015) already start with the task description and its terminology, or even terminologies in general. Other researchers framed this by saying that "working on ill-structured problems or problems with multiple weak dimensions requires more demanding information activities" (Palmer 2006).…”

“…A question in our concrete research project (Kade et al 2015) was: who should lead the team with the task of creating a new head-mounted display? The designer, the engineer or the computer scientist, or maybe a forth person, trained in project management?…”

Tracking and classification of head movement by inertial measurement unit data

“…Область их применения начинается от развлекательных [1] и образовательных [2] целей и заканчивается применением в промышленности [3] и военном деле [4]. Подобные шлемы яв-ляются крайне удобным и перспективным интерфейсом для взаимодействия с пользователем и все чаще применяются в робототехнике [5,6].…”