Architectures and Codecs for Real-Time Light Field Streaming

Abstract

Light field 3D displays represent a major step forward in visual realism, providing glasses-free spatial vision of real or virtual scenes. Applications that capture and process live imagery have to process data captured by potentially tens to hundreds of cameras and control tens to hundreds of projection engines making up the human perceivable 3D light field using a distributed processing system. The associated massive data processing is difficult to scale beyond a specific number and resolution of images, limited by the capabilities of the individual computing nodes. We therefore analyze the bottlenecks and data flow of the light field conversion process and identify possibilities to introduce better scalability. Based on this analysis we propose two different architectures for distributed light field processing. To avoid using uncompressed video data all along the processing chain, we also analyze how the operation of the proposed architectures can be supported by existing image / video codecs.