Real-time pixel luminance optimization for dynamic multi-projection mapping

Abstract

Using projection mapping enables us to bring virtual worlds into shared physical spaces. In this paper, we present a novel, adaptable and real-time projection mapping system, which supports multiple projectors and high quality rendering of dynamic content on surfaces of complex geometrical shape. Our system allows for smooth blending across multiple projectors using a new optimization framework that simulates the diffuse direct light transport of the physical world to continuously adapt the color output of each projector pixel. We present a real-time solution to this optimization problem using off-the-shelf graphics hardware, depth cameras and projectors. Our approach enables us to move projectors, depth camera or objects while maintaining the correct illumination, in realtime, without the need for markers on the object. It also allows for projectors to be removed or dynamically added, and provides compelling results with only commodity hardware.

Recent Advances in Projection Mapping Algorithms, Hardware and Applications

“…This figure summarizes examples of rigid dynamic projection mapping: The top row shows three augmentation results of a rigid dynamic multi‐projector mapping onto a uniform gray face statue [SCT*15]. In the bottom left, a dynamic projection mapping onto an animatronic head is shown using multiple projectors which are compensating for sub‐surface scattering as well as image degradation artifacts from defocus [BBG*13].…”

“…A method which uses a low‐resolution online‐reconstruction for projector registration was presented [rkk14]: The shape of an augmented object is measured on‐line by triangulation using projected features and the corresponding camera pixel correspondences, then the iterative closest points (ICP) algorithm [Zha94] is used to estimate the six degrees of freedom (6DOF) movement which allows to register the projection to the current pose of the real object to augment. Another research group optimized projection images by solving the light transport matrix, which was derived from the 6DOF relations between each projector and the object measured by a Kinect depth sensor [SCT*15] (cf. Figure ).…”

Adaptive stray-light compensation in dynamic multi-projection mapping

“…We use the system presented by Siegl et al [1] as a basis for this work. Their system is able to solve the complex problem of blending multiple projectors on an arbitrary target geometry in real time.…”

“…Before sending the final color to the projector, we de-linearize the colors by applying gamma correction. For a more detailed discussion we refer the reader to Siegl et al [1]. …”

“…The system presented in this paper is based on the work by Siegl et al [1]. Their multi-projection system tracking the target object in blending between projectors is continuously adapted to the current object position and orientation.…”