Learning to Capture Light Fields Through a Coded Aperture Camera

Inagaki, Yasuyoshi; Kobayashi, Yasuhiro; Takahashi, Keita; Fujii, Toshiaki; Nagahara, Hajime

doi:10.1007/978-3-030-01234-2_26

Cited by 59 publications

(118 citation statements)

References 37 publications

Supporting

Mentioning

115

Contrasting

Order By: Relevance

“…As we can see from the error maps shown in the figure and Table II, for the challenging case of single coded image (N = 1), our approach outperforms Inagaki et al [11] by 2 dB on the testset. The red and blue insets shown in the figure detail the top-right view reconstructed along with the EPIs for N = 1 case.…”

Section: B Coded Aperture Light Field Reconstructionmentioning

confidence: 66%

“…This ensures that the data has variety of relative defocus w.r.t the focal plane during training. We compared our reconstructions with both direct regression (Nabati et al [20], Inagaki et al [11]) and disparity based rendering from four corner views of LF (Kalantari et al [6]) and from a single image (Srinivasan et al [18]). Our CLF and focus-defocus approaches perform as well as the four corner view method of Kalantari et al and outperform direct regression and single image based approaches.…”

Section: Discussionmentioning

confidence: 99%

“…We show our reconstructed centerview from the coded image along with the disparity maps. We compare our results with the learning approach of Inagaki et al [11] which reconstructs LF from coded images directly without any disparity estimation. For comparison, we use the coded aperture masks provided by Inagaki et al for the case of single (N = 1) and dual (N = 2) coded aperture at 8 × 8 angular resolution and resize them to 7 × 7.…”

Section: B Coded Aperture Light Field Reconstructionmentioning

confidence: 99%

See 2 more Smart Citations

A Unified Learning-Based Framework for Light Field Reconstruction From Coded Projections

Vadathya

Girish

Mitra

2020

IEEE Trans. Comput. Imaging

View full text Add to dashboard Cite

Light fields present a rich way to represent the 3D world by capturing the spatio-angular dimensions of the visual signal. However, the popular way of capturing light fields (LF) via a plenoptic camera presents a spatio-angular resolution tradeoff. Computational imaging techniques such as compressive light field and programmable coded aperture reconstruct full sensor resolution LF from coded projections obtained by multiplexing the incoming spatio-angular light field. Here, we present a unified learning framework that can reconstruct LF from a variety of multiplexing schemes with minimal number of coded images as input. We consider three light field capture schemes: heterodyne capture scheme with code placed near the sensor, coded aperture scheme with code at the camera aperture and finally the dual exposure scheme of capturing a focus-defocus pair where there is no explicit coding. Our algorithm consists of three stages 1) we recover the all-in-focus image from the coded image 2) we estimate the disparity maps for all the LF views from the coded image and the all-in-focus image, 3) we then render the LF by warping the all-in-focus image using disparity maps. We show that our proposed learning algorithm performs either on par with or better than the state-of-the-art methods for all the three multiplexing schemes. LF from focus-defocus pair is especially attractive as it requires no hardware modification and produces LF reconstructions that are comparable to the current state of the art learning-based view synthesis approaches from multiple images. Thus, our work paves the way for capturing full-resolution LF using conventional cameras such as DSLRs and smartphones.Index Terms-Light field resolution trade-off, compressive light field imaging, coded aperture photography, disparity based view synthesis.

show abstract

Section: B Coded Aperture Light Field Reconstructionmentioning

confidence: 66%

Section: Discussionmentioning

confidence: 99%

Section: B Coded Aperture Light Field Reconstructionmentioning

confidence: 99%

See 1 more Smart Citation

A Unified Learning-Based Framework for Light Field Reconstruction From Coded Projections

Vadathya

Girish

Mitra

2020

IEEE Trans. Comput. Imaging

View full text Add to dashboard Cite

show abstract

“…distribution can uniquely recover x in Equation (4.9) [171,172,173]. It is also possible to find the optimal Φ using learning-based methods, which is an ongoing research problem [174,175,176].…”

Section: Compressive Sensingmentioning

confidence: 99%

“…Three light field video data sets were used for this purpose: Boxer-Gladiator-Irish, Chess, and Chess-moving from [211]. We compared the result of reconstruction using SM3 with temporal window size of 3 (β = 3), with respect to the method of Marwah et al [22], Miandji et al [23], and the deep [22], Miandji et al [23], and Inagaki et al [176].…”

Section: Sensing Model 3 (Sm3)mentioning

confidence: 99%