Comparison of Layer Operations and Optimization Methods for Light Field Display

Abstract

A light-field display provides not only binocular depth sensation but also natural motion parallax with respect to head motion, which invokes a strong feeling of immersion. Such a display can be implemented with a set of stacked layers, each of which has pixels that can carry out light-ray operations (multiplication and addition). With this structure, the appearance of the display varies over the observed directions (i.e., a light field is produced) because the light rays pass through different combinations of pixels depending on both the originating points and outgoing directions. To display a specific 3-D scene, these layer patterns should be optimized to produce a light field that is as close as possible to that produced by the target three-dimensional scene. To deepen the understanding for this type of light field display, we focused on two important factors: light-ray operations carried out using layers and optimization methods for the layer patterns. Specifically, we compared multiplicative and additive layers, which are optimized using analytical methods derived from mathematical optimization or faster data-driven methods implemented as convolutional neural networks (CNNs). We compared combinations within these two factors in terms of the accuracy of light-field reproduction and computation time. Our results indicate that multiplicative layers achieve better accuracy than additive ones, and CNN-based methods perform faster than the analytical ones. We suggest that the best choice in terms of the balance between accuracy and computation speed is using multiplicative layers optimized using a CNN-based method.

A Flexible Coding Scheme Based on Block Krylov Subspace Approximation for Light Field Displays with Stacked Multiplicative Layers

“…Here, we assume that a light-field display is composed of three layers located at , where . It must be noted that z corresponds to the disparity among the directional views, rather than the physical length [ 14 ]. The multiplicative layers need to be optimized to display a 3-D scene.…”

“…Multi-layered or tensor light field displays offer an optimized solution to support direction-dependent outputs simultaneously, without sacrificing the resolution in reproducing dense light fields. This is deemed as a critical characteristic of glasses-free multi-view displays [ 5 , 11 , 12 , 13 , 14 , 15 , 16 , 17 ]. A typical structure of a layered 3D display is demonstrated in Figure 1 a.…”

“…The transmittance of pixels on each layer can be controlled independently by carrying out light-ray operations (multiplication and addition) on the layered patterns. A multi-layered display with liquid crystal display (LCD) panels and a backlight is implemented by performing multiplicative operations on layers, and additive layers are fabricated with holographic optical elements (HOEs) and projectors [ 14 ]. With this structure, the layer patterns allow light rays to pass through different combinations of pixels depending on the viewing directions.…”

“…The CNN is employed to optimize multiplicative layers obtained from light field data. CNN-based methods are proven to be better in terms of balance between computation time and accuracy than the previous analytical optimization approaches based on non-negative tensor factorization [ 5 , 14 , 17 , 18 ]. A novel coding scheme for the multiplicative layers is proposed based on a randomized block Krylov singular value decomposition framework.…”

Render‐based factorization for additive light field display

“…But the performance improvement was only demonstrated for offline rendering scenarios. Maruyama et al 32 compared multiplicative and additive layers in terms of the accuracy of light-field reproduction and computation time.…”