Time-Multiplexed Coded Aperture Imaging: Learned Coded Aperture and Pixel Exposures for Compressive Imaging Systems

Vargas, Edwin; Martel, Julien N. P.; Wetzstein, Gordon; Argüello, Henry

doi:10.1109/iccv48922.2021.00269

Cited by 21 publications

(16 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We noticed that Vargas et al [42] recently proposed an imaging architecture similar to ours for compressive light field acquisition. However, their method was designed for static light fields.…”

Section: Computational Photographymentioning

confidence: 76%

Acquiring a Dynamic Light Field through a Single-Shot Coded Image

Mizuno¹,

Takahashi²,

Yoshida³

et al. 2022

Preprint

View full text Add to dashboard Cite

We propose a method for compressively acquiring a dynamic light field (a 5-D volume) through a single-shot coded image (a 2-D measurement). We designed an imaging model that synchronously applies aperture coding and pixel-wise exposure coding within a single exposure time. This coding scheme enables us to effectively embed the original information into a single observed image. The observed image is then fed to a convolutional neural network (CNN) for light-field reconstruction, which is jointly trained with the camera-side coding patterns. We also developed a hardware prototype to capture a real 3-D scene moving over time. We succeeded in acquiring a dynamic light field with 5×5 viewpoints over 4 temporal sub-frames (100 views in total) from a single observed image. Repeating capture and reconstruction processes over time, we can acquire a dynamic light field at 4× the frame rate of the camera. To our knowledge, our method is the first to achieve a finer temporal resolution than the camera itself in compressive light-field acquisition. Our software is available from our project webpage. 1

show abstract

Section: Computational Photographymentioning

confidence: 76%

Acquiring a Dynamic Light Field through a Single-Shot Coded Image

Mizuno¹,

Takahashi²,

Yoshida³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…To enhance the coding capability of the CA method, time-multiplexed coded aperture (TMCA) [19], [20] (called factorized modulation in [19]) was introduced. TMCA can be implemented by combining CA and pixel-wise exposure coding [22], [23], which are synchronously varied during an exposure.…”

Section: Coded Aperture To Time-multiplexed Coded Aperturementioning

confidence: 99%

“…The entire pipeline is trained end to end so that the difference between the input LF and output LF is minimized. This kind of deep-learning-based approach has proven to be quite successful for compressive LF imaging tasks [14], [20], [27].…”

Section: Algorithm Pipelinementioning

confidence: 99%

“…A possible solution to overcome this limitation is to exploit rapid optical control during a single exposure. Specifically, we consider two promising methods with this kind of rapidly controlled imaging: time-multiplexed coded aperture (TMCA) [19], [20] and coded focal stack (CFS) [21], in which the CA and FS methods, respectively, are combined with pixel-wise exposure coding within a single exposure.…”

Section: Introductionmentioning

confidence: 99%

“…Note that TMCA and CFS were proposed individually in previous works (TMCA: [19], [20]; CFS: [21]), and CFS was not directly used for compressive LF imaging. To our knowledge, we are the first to present an overall discussion of TMCA and CFS and to compare and validate their effectiveness in the context of compressive LF imaging.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Time-Multiplexed Coded Aperture and Coded Focal Stack -Comparative Study on Snapshot Compressive Light Field Imaging

Tateishi

Tsutake

Takahashi

et al. 2022

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

A light field (LF), which is represented as a set of dense, multi-view images, has been used in various 3D applications. To make LF acquisition more efficient, researchers have investigated compressive sensing methods by incorporating certain coding functionalities into a camera. In this paper, we focus on a challenging case called snapshot compressive LF imaging, in which an entire LF is reconstructed from only a single acquired image. To embed a large amount of LF information in a single image, we consider two promising methods based on rapid optical control during a single exposure: time-multiplexed coded aperture (TMCA) and coded focal stack (CFS), which were proposed individually in previous works. Both TMCA and CFS can be interpreted in a unified manner as extensions of the coded aperture (CA) and focal stack (FS) methods, respectively. By developing a unified algorithm pipeline for TMCA and CFS, based on deep neural networks, we evaluated their performance with respect to other possible imaging methods. We found that both TMCA and CFS can achieve better reconstruction quality than the other snapshot methods, and they also perform reasonably well compared to methods using multiple acquired images. To our knowledge, we are the first to present an overall discussion of TMCA and CFS and to compare and validate their effectiveness in the context of compressive LF imaging.

show abstract

Exposure-Aware Dynamic Weighted Learning for Single-Shot HDR Imaging

Vien

Lee

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Time-Multiplexed Coded Aperture Imaging: Learned Coded Aperture and Pixel Exposures for Compressive Imaging Systems

Cited by 21 publications

References 48 publications

Acquiring a Dynamic Light Field through a Single-Shot Coded Image

Acquiring a Dynamic Light Field through a Single-Shot Coded Image

Time-Multiplexed Coded Aperture and Coded Focal Stack -Comparative Study on Snapshot Compressive Light Field Imaging

Exposure-Aware Dynamic Weighted Learning for Single-Shot HDR Imaging

Contact Info

Product

Resources

About