Deep fully-connected networks for video compressive sensing

Iliadis, Michael; Spinoulas, Leonidas; Katsaggelos, Aggelos K.

doi:10.1016/j.dsp.2017.09.010

Cited by 198 publications

(130 citation statements)

References 34 publications

Supporting

Mentioning

120

Contrasting

Order By: Relevance

“…We are not aiming to compete with these algorithms as they are usually complicated and require data to train the neural networks. Furthermore, some of these algorithms require the sensing matrix being spatially repetitive [30], which is very challenging (or unrealistic) in real applications. In addition, we have noticed that limited improvement (around 2dB) has been obtained using these deep learning techniques in [30] compared with GMM.…”

Section: Related Work and Organization Of This Papermentioning

confidence: 99%

“…Furthermore, some of these algorithms require the sensing matrix being spatially repetitive [30], which is very challenging (or unrealistic) in real applications. In addition, we have noticed that limited improvement (around 2dB) has been obtained using these deep learning techniques in [30] compared with GMM. By contrast, our proposed algorithm has improved the results significantly (> 4dB) over GMM.…”

Section: Related Work and Organization Of This Papermentioning

confidence: 99%

“…The performance of GAP-wavelet proposed in [6] is similar to GAP-TV as demonstrated in [17] and thus omitted here. Due to the special requirements of the mask in deep learning based methods [30], we do not compare with them. However, as mentioned earlier, we can notice that limited improvement (around 2 dB) has been obtained using these deep learning techniques in [30] [57] are employed to evaluate the quality of reconstructed videos.…”

Section: Video Snapshot Compressive Imagingmentioning

confidence: 99%

See 2 more Smart Citations

Rank Minimization for Snapshot Compressive Imaging

Liu

Yuan

Suo

et al. 2019

IEEE Trans. Pattern Anal. Mach. Intell.

301

200

View full text Add to dashboard Cite

Snapshot compressive imaging (SCI) refers to compressive imaging systems where multiple frames are mapped into a single measurement, with video compressive imaging and hyperspectral compressive imaging as two representative applications. Though exciting results of high-speed videos and hyperspectral images have been demonstrated, the poor reconstruction quality precludes SCI from wide applications. This paper aims to boost the reconstruction quality of SCI via exploiting the high-dimensional structure in the desired signal. We build a joint model to integrate the nonlocal self-similarity of video/hyperspectral frames and the rank minimization approach with the SCI sensing process. Following this, an alternating minimization algorithm is developed to solve this non-convex problem. We further investigate the special structure of the sampling process in SCI to tackle the computational workload and memory issues in SCI reconstruction. Both simulation and real data (captured by four different SCI cameras) results demonstrate that our proposed algorithm leads to significant improvements compared with current state-of-the-art algorithms. We hope our results will encourage the researchers and engineers to pursue further in compressive imaging for real applications.Index Terms-Compressive sensing, computational imaging, coded aperture, image processing, video processing, nuclear norm, rank minimization, low rank, hyperspectral images, coded aperture snapshot spectral imaging (CASSI), coded aperture compressive temporal imaging (CACTI).

show abstract

Section: Related Work and Organization Of This Papermentioning

confidence: 99%

Section: Related Work and Organization Of This Papermentioning

confidence: 99%

Section: Video Snapshot Compressive Imagingmentioning

confidence: 99%

See 1 more Smart Citation

Rank Minimization for Snapshot Compressive Imaging

Liu

Yuan

Suo

et al. 2019

IEEE Trans. Pattern Anal. Mach. Intell.

301

200

View full text Add to dashboard Cite

show abstract

“…Our paper investigates the temporal upsampling problem. While previous approaches investigate in the framework of compressive sensing [1,14,17,26,35,38,41], we formulate our work as fusing event streams with intensity images to obtain a temporally dense video. Compared to existing literature [36] which integrates event counts per pixel across time, our differentiable model utilizes "tanh" functions as event activation units and imposes sparsity constraints on both spatial and temporal domain.…”

Section: Related Workmentioning

confidence: 99%

Event-Driven Video Frame Synthesis

Wang¹,

Jiang²,

He³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

Temporal Video Frame Synthesis (TVFS) aims at synthesizing novel frames at timestamps different from existing frames, which has wide applications in video codec, editing and analysis. In this paper, we propose a high framerate TVFS framework which takes hybrid input data from a lowspeed frame-based sensor and a high-speed event-based sensor. Compared to frame-based sensors, event-based sensors report brightness changes at very high speed, which may well provide useful spatio-temoral information for high framerate TVFS. In our framework, we first introduce a differentiable forward model to approximate the physical sensing process, fusing the two different modes of data as well as unifying a variety of TVFS tasks, i.e., interpolation, prediction and motion deblur. We leverage autodifferentiation which propagates the gradients of a loss defined on the measured data back to the latent high framerate video. We show results with better performance compared to state-ofthe-art. Second, we develop a deep learning-based strategy to enhance the results from the first step, which we refer as a residual "denoising" process. Our trained "denoiser" is beyond Gaussian denoising and shows properties such as contrast enhancement and motion awareness. We show that our framework is capable of handling challenging scenes including both fast motion and strong occlusions. Supplementary material, demo and code are released at: https://github.com/winswang/int-event-fusion/tree/win10.

show abstract

“…14 The latest compressive video sensing research learned a linear mapping between video sequences and corresponding measured frames. 15 In addition, the correlation between consecutive frames in the frequency domain 16 and other transform domains 17 was also used.…”

Section: Introductionmentioning

confidence: 99%

Nonconvex compressive video sensing

Chen

Yan

Cui

et al. 2016

J. Electron. Imaging

View full text Add to dashboard Cite

High-speed cameras explore more details than normal cameras in the time sequence, while the conventional video sampling suffers from the trade-off between temporal and spatial resolutions due to the sensor’s physical limitation. Compressive sensing overcomes this obstacle by combining the sampling and compression procedures together. A single-pixel-based real-time video acquisition is proposed to record dynamic scenes, and a fast nonconvex algorithm for the nonconvex sorted ℓ1 regularization is applied to reconstruct frame differences using few numbers of measurements. Then, an edge-detection-based denoising method is employed to reduce the error in the frame difference image. The experimental results show that the proposed algorithm together with the single-pixel imaging system makes compressive video cameras available.

show abstract

Deep fully-connected networks for video compressive sensing

Cited by 198 publications

References 34 publications

Rank Minimization for Snapshot Compressive Imaging

Rank Minimization for Snapshot Compressive Imaging

Event-Driven Video Frame Synthesis

Nonconvex compressive video sensing

Contact Info

Product

Resources

About