High-speed imaging using CMOS image sensor with quasi pixel-wise exposure

Nagahara, Hajime; Sonoda, T.; Endo, Kenta; Sugiyama, Y.; Taniguchi, Rin-ichiro

doi:10.1109/iccphot.2016.7492875

Cited by 21 publications

(18 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Increasing the compression ratio is also important because it can extend the observation time or depth range. Although a compression ratio of 8× was used in this study, a compression ratio of 32 has been demonstrated in [9,39] based on dictionary or learning-based methods. For specialized applications, such as TOF depth imaging, these methods can be efficient.…”

Section: Discussionmentioning

confidence: 99%

“…Articles on compressive image sensors based on the image compression by analog or digital circuits are reviewed in [28]. The authors of [9,10] adopt a different signal compression scheme where the signal is compressed in the charge domain in pixel. This scheme is advantageous in temporal compression in terms of pixel size and power consumption.…”

Section: Compressive Sensing and Imagingmentioning

confidence: 99%

“…We proposed a multi-aperture UHS image sensor that adopted an entirely different scheme using compressive sensing, which was motivated by computational photography [7,8]. Compressive video is a similar technique, although the frame rates are close to video rates, namely, 30 fps [9,10]. In this scheme, input optical images are compressed in the charge domain as an inner-product or correlation with a random, temporally coded shutter at each pixel.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Dual-Mode 303-Megaframes-per-Second Charge-Domain Time-Compressive Computational CMOS Image Sensor

Kagawa

Horio

Pham

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

An ultra-high-speed computational CMOS image sensor with a burst frame rate of 303 megaframes per second, which is the fastest among the solid-state image sensors, to our knowledge, is demonstrated. This image sensor is compatible with ordinary single-aperture lenses and can operate in dual modes, such as single-event filming mode or multi-exposure imaging mode, by reconfiguring the number of exposure cycles. To realize this frame rate, the charge modulator drivers were adequately designed to suppress the peak driving current taking advantage of the operational constraint of the multi-tap charge modulator. The pixel array is composed of macropixels with 2 × 2 4-tap subpixels. Because temporal compressive sensing is performed in the charge domain without any analog circuit, ultrafast frame rates, small pixel size, low noise, and low power consumption are achieved. In the experiments, single-event imaging of plasma emission in laser processing and multi-exposure transient imaging of light reflections to extend the depth range and to decompose multiple reflections for time-of-flight (TOF) depth imaging with a compression ratio of 8× were demonstrated. Time-resolved images similar to those obtained by the direct-type TOF were reproduced in a single shot, while the charge modulator for the indirect TOF was utilized.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Compressive Sensing and Imagingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Dual-Mode 303-Megaframes-per-Second Charge-Domain Time-Compressive Computational CMOS Image Sensor

Kagawa

Horio

Pham

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…In a coded exposure, a shutter function turns individual pixels "on" or "off" during the exposure, thus modulating light integration. Pixel-wise shutters can be realized electronically modifying the pixel architecture [34,53,32,50] or using spatial light modulators (SLMs) [20]. More specifically, we generate a TMCA by synchronizing a CA that changes its pattern in time -dubbed as a time-varying CA-and a spatially varying shutter-function realizing a coded exposure.…”

Section: Introductionmentioning

confidence: 99%

Time-Multiplexed Coded Aperture Imaging: Learned Coded Aperture and Pixel Exposures for Compressive Imaging Systems

Vargas¹,

Martel²,

Wetzstein³

et al. 2021

Preprint

View full text Add to dashboard Cite

Compressive imaging using coded apertures (CA) is a powerful technique that can be used to recover depth, light fields, hyperspectral images and other quantities from a single snapshot. The performance of compressive imaging systems based on CAs mostly depends on two factors: the properties of the mask's attenuation pattern, that we refer to as "codification" and the computational techniques used to recover the quantity of interest from the coded snapshot. In this work, we introduce the idea of using time-varying CAs synchronized with spatially varying pixel shutters. We divide the exposure of a sensor into sub-exposures at the beginning of which the CA mask changes and at which the sensor's pixels are simultaneously and individually switched "on" or "off". This is a practically appealing codification as it does not introduce additional optical components other than the already present CA but uses a change in the pixel shutter that can be easily realized electronically. We show that our proposed time multiplexed coded aperture (TMCA) can be optimized end-to-end and induces better coded snapshots enabling superior reconstructions in two different applications: compressive light field imaging and hyperspectral imaging. We demonstrate both in simulation and on real captures (taken with prototypes we built) that this codification outperforms the state-of-the-art compressive imaging systems by more than 4dB in those applications.

show abstract

“…Analysis of theoretical and experimental properties of compressive video cameras which utilize per-pixel coded exposure sequences has been done as well [5] - [7]. FPA implementations using programmable coded exposure sequences have also been developed [11], [12]. All of these algorithms work with an approach to maximize the information content in space and time using a compressed sensing approach which assumes the signal can be sparsely represented in a transform domain.…”

Section: Introductionmentioning

confidence: 99%

An Adaptive Video Acquisition Scheme for Object Tracking and its Performance Optimization

Banerjee¹,

Chopp²,

Serra³

et al. 2021

Preprint

View full text Add to dashboard Cite

We present a novel adaptive host-chip modular architecture for video acquisition to optimize an overall objective task constrained under a given bit rate. The chip is a high resolution imaging sensor such as gigapixel focal plane array (FPA) with low computational power deployed on the field remotely, while the host is a server with high computational power. The communication channel data bandwidth between the chip and host is constrained to accommodate transfer of all captured data from the chip. The host performs objective task specific computations and also intelligently guides the chip to optimize (compress) the data sent to host. This proposed system is modular and highly versatile in terms of flexibility in re-orienting the objective task. In this work, object tracking is the objective task. While our architecture supports any form of compression/distortion, in this paper we use quadtree (QT)-segmented video frames. We use Viterbi (Dynamic Programming) algorithm to minimize the area normalized weighted rate-distortion allocation of resources. The host receives only these degraded frames for analysis. An object detector is used to detect objects, and a Kalman Filter based tracker is used to track those objects. Evaluation of system performance is done in terms of Multiple Object Tracking Accuracy (MOTA) metric. In this proposed novel architecture, performance gains in MOTA is obtained by twice training the object detector with different system generated distortions as a novel 2-step process. Additionally, object detector is assisted by tracker to upscore the region proposals in the detector to further improve the performance.

show abstract

High-speed imaging using CMOS image sensor with quasi pixel-wise exposure

Cited by 21 publications

References 18 publications

A Dual-Mode 303-Megaframes-per-Second Charge-Domain Time-Compressive Computational CMOS Image Sensor

A Dual-Mode 303-Megaframes-per-Second Charge-Domain Time-Compressive Computational CMOS Image Sensor

Time-Multiplexed Coded Aperture Imaging: Learned Coded Aperture and Pixel Exposures for Compressive Imaging Systems

An Adaptive Video Acquisition Scheme for Object Tracking and its Performance Optimization

Contact Info

Product

Resources

About