Adaptive stray-light compensation in dynamic multi-projection mapping

Abstract

Projection-based mixed reality is an effective tool to create immersive visualizations on real-world objects. Its wide range of applications includes art installations, education, stage shows, and advertising. In this work, we enhance a multi-projector system for dynamic projection mapping by handling various physical stray-light effects: interreflection, projector black-level, and environmental light in real time for dynamic scenes. We show how all these effects can be efficiently simulated and accounted for at runtime, resulting in significantly improved projection mapping results. By adding a global optimization step, we can further increase the dynamic range of the projection.

CompenNet++: End-to-End Full Projector Compensation

“…Recently, Siegl et al [51], [52] perform full compensation on nonplanar Lambertian surfaces for dynamic real-time projection mapping. Similar to [17], they assume the target objects are white and textureless.…”

“…In theory, the projector compensation process is a very complicated nonlinear function involving the camera and the projector sensor radiometric responses [38], lens distortion/vignetting [30], perspective transformations [23], [63], surface material reflectance [21], [41], defocus [31], [59], [61] and inter-reflection [53]. A great amount of effort has been dedicated to designing practical and accurate compensation models, which can be roughly categorized into two types: full compensation [4], [17], [44], [49], [51], [52], [58] and partial ones [1], [3], [13], [15], [20], [35], [38], [48], [53].…”

“…W ITH the recent advance in projector technologies, projectors have been gaining increasing popularity with many applications [1], [4], [12], [13], [16], [21], [22], [37], [43], [51], [52], [55], [60]. Existing systems typically require the projection surface (screen) to be planar, white and textureless, and under reasonable environment illumination.…”

“…These requirements often create bottlenecks for generalization of projector systems. Full projector geometric correction and photometric compensation [2], [4], [17], [44], [51], [52] aims to address this issue by modifying a projector input image to compensate for the projection setup geometry [5], [37], [42], [43], [54] and associated photometric environment [1], [3], [15], [20], [60]. In the rest of the text, we call it full compensation for conciseness.…”

Recent Advances in Projection Mapping Algorithms, Hardware and Applications

“…Most of these methods were also able to compute a compensation image in real‐time using GPU‐based parallel processing. A more recent publication solving the same problem in real‐time for multiple projectors in a dynamic environment has been presented by Siegl et al [SCSB17]. Habe et al presented an inter‐reflection compensation algorithm specifically designed for dome‐shaped projection surfaces [HSM07].…”