Recently, the JPEG standardization committee created an initiative called JPEG Pleno. "Pleno" is a reference to "plenoptic," a mathematical representation that not only provides information about any point within a scene but also about how it changes when observed from different positions. "Pleno" is also the Latin word for "complete," a reference to the JPEG committee's desire for future imaging to provide a more complete description of scenes, well beyond what's possible today. Here, we discuss the rationale behind the vision for the JPEG Pleno initiative and describe how it can potentially reinvent the future of imaging.

Up to now, an increase in camera resolution required image sensors with more and more pixels. However, acquisition systems are limited in their pixels per second throughput given as power and complexity constraints. Simply capturing more pixels in a given system is often not possible. We propose a new non-regular imaging architecture that samples only few pixels and reconstructs a high resolution image afterwards. Our sampling is optimized to provide non-regular spatial sampling from a sensor with regular readout circuits. An existing slow image acquisition system can then be used to capture the data. The image reconstruction is performed with a local sparsity-based approach. The result is a high resolution image that requires a much smaller effort during acquisition

We present a new method for capturing high dynamic range video (HDRV). Our method is based on spatially varying exposures, where individual pixels are covered with filters for different optical attenuation. For preventing the loss in resolution we use a new non- regular arrangement of the attenuation pattern. Subsequent image reconstruction based on the sparsity assumption allows the recon- struction of natural images with high detail. Index Terms High Dynamic Range Image Sensor, Digital Camera, Resolution Enhancement, Sparsit

Plenoptic cameras based on micro lens arrays as well as multi aperture cameras are able to capture a multitude of images with slightly shifted viewpoints. Although the amount of parallax between adjacent views is limited, precautions have to be taken in order to avoid alias when performing direct lightfield rendering. Against this background, we present an approach for the dense reconstruction of a lightfield based on a sparse lightfield acquired from a multi aperture camera with subsequent disparity estimation and depth image based view interpolation. Results show that the approach is suitable for all-in-focus-rendering

In high quality imaging even tiny distortions as small as a single pixel are visible and can not be accepted. Although the production quality of CMOS image sensors is very high, for reasonable yields we still need to accept some defect pixels and clusters of defects in large image sensors. In this paper we will compare compensation algorithms for raw image sensor data. We propose a new approach based on the sparsity assumption that outperforms existing defect compensation algorithms. Furthermore, our proposed interpolation algorithm is universal and not at all adapted to Bayer pattern images. It can directly be applied to any regular color filter pattern or gray scale image. Our examples show, that image sensors with large clusters of defects can still be used for the generation of high quality images

No abstract

No abstract

Post-production technologies like visual effects (VFX) or digital grading are essential for visual storytelling in today's movie production. Tremendous efforts are made to seamlessly integrate computer generated (CG) elements into live action plates, to refine looks and to provide various delivery formats such as Stereo 3D (S3D). Thus, additional tools to assist and improve element integration, grading and S3D techniques could ease complicated, time consuming, manual and costly processes. Geometric data like depth information is a key for these tasks but a typical main unit camera shoot does not deliver this information. Although e.g. light detection and ranging (LIDAR) scans are used to capture 3D set information, frame by frame geometric information for dynamic scenes including moving camera, actors and props are not being recorded. Stereo camera systems deliver additional data for depth map generation but accuracy is limited. This work suggests a method of capturing light field data within a regular live action shoot. We compute geometric data for post-production use cases such as relighting, depth-based compositing, 3D integration, virtual camera, digital focus as well as virtual backlots. Thus, the 2D high quality life action plate shot by the Director of Photography (DoP) comes along with image data from the light field array that can be further processed and utilized. As a proof-of-concept we produced a fictitious commercial using a main camera and a multi-camera array fitted into an S3D mirror rig. For this reference movie, the image of the center array cam has been selected for further image manipulation. The quality of the depth information is evaluated on simulated data and live action footage. Our experiments show that quality of depth maps depends on array geometry and scene setup