Snapshot Compressed Sensing: Performance Bounds and Algorithms

Jalali, Shirin; Yuan, Xin

doi:10.1109/tit.2019.2940666

Cited by 95 publications

(45 citation statements)

References 62 publications

(115 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where Φ t = Diag(vec(M(:, :, t))) is the diagonal matrix with vec(M(:, :, t)) as the diagonal elements. Note that Φ is a very sparse matrix and the theoretical bounds of VCS have been developed in [8].…”

Section: Grayscale Video Compressive Sensingmentioning

confidence: 99%

Two-Stage is Enough: A Concise Deep Unfolding Reconstruction Network for Flexible Video Compressive Sensing

Yang¹,

Yuan²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

We consider the reconstruction problem of video compressive sensing (VCS) under the deep unfolding/rolling structure. Yet, we aim to build a flexible and concise model using minimum stages. Different from existing deep unfolding networks used for inverse problems, where more stages are used for higher performance but without flexibility to different masks and scales, hereby we show that a 2-stage deep unfolding network can lead to the state-of-theart (SOTA) results (with a 1.7dB gain in PSNR over the single stage model, RevSCI [3]) in VCS. The proposed method possesses the properties of adaptation to new masks and ready to scale to large data without any additional training thanks to the advantages of deep unfolding. Furthermore, we extend the proposed model for color VCS to perform joint reconstruction and demosaicing. Experimental results demonstrate that our 2-stage model has also achieved SOTA on color VCS reconstruction, leading to a >2.3dB gain in PSNR over the previous SOTA algorithm based on plugand-play framework, meanwhile speeds up the reconstruction by >17 times. In addition, we have found that our network is also flexible to the mask modulation and scale size for color VCS reconstruction so that a single trained network can be applied to different hardware systems. The code and models will be released to the public.

show abstract

Section: Grayscale Video Compressive Sensingmentioning

confidence: 99%

Two-Stage is Enough: A Concise Deep Unfolding Reconstruction Network for Flexible Video Compressive Sensing

Yang¹,

Yuan²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The real challenge is the inverse problem, i.e., the decoder or reconstruction algorithms. 22 More specifically, given the compressed measurement y and sensing matrix Φ.…”

Section: Mathematical Model Of Scimentioning

confidence: 99%

Revisit Dictionary Learning for Video Compressive Sensing under the Plug-and-Play Framework

Yang¹,

Zhao²

2021

Preprint

View full text Add to dashboard Cite

Aiming at high-dimensional (HD) data acquisition and analysis, snapshot compressive imaging (SCI) obtains the 2D compressed measurement of HD data with optical imaging systems and reconstructs HD data using compressive sensing algorithms. While the Plug-and-Play (PnP) framework offers an emerging solution to SCI reconstruction, its intrinsic denoising process is still a challenging problem. Unfortunately, existing denoisers in the PnP framework either suffer limited performance or require extensive training data. In this paper, we propose an efficient and effective shallow-learning-based algorithm for video SCI reconstruction. Revisiting dictionary learning methods, we empower the PnP framework with a new denoiser, the kernel singular value decomposition (KSVD). Benefited from the advent of KSVD, our algorithm retains a good trade-off among quality, speed, and training difficulty. On a variety of datasets, both quantitative and qualitative evaluations of our simulation results demonstrate the effectiveness of our proposed method. In comparison to a typical baseline using total variation, our method achieves around 2 dB improvement in PSNR and 0.2 in SSIM. We expect that our proposed PnP-KSVD algorithm can serve as a new baseline for video SCI reconstruction.

show abstract

“…sensing matrix, x ∈ R nB is the 3D data (by vectorizing each frame and stacking them), and e is the measurement noise; here B denotes that every B video frames are collapsed into a single 2D measurement. Though algorithms have been fully developed to reconstruct the video from its snapshot measurement in recent years, the fundamental issue remains: this inverse problem is inherently ill-posed, which makes the recovery of the signal x inaccurate and unstable for noise-affected data y [12]. The rapid advancement of deep learning and artificial intelligence have empowered a new wave of revolutionary solutions towards these previously intractable problems.…”

Section: Introductionmentioning

confidence: 99%

Deep Equilibrium Models for Video Snapshot Compressive Imaging

Zhao¹,

Yuan²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

The ability of snapshot compressive imaging (SCI) systems to efficiently capture high-dimensional (HD) data has led to an inverse problem, which consists of recovering the HD signal from the compressed and noisy measurement. While reconstruction algorithms grow fast to solve it with the recent advances of deep learning, the fundamental issue of accurate and stable recovery remains. To this end, we propose deep equilibrium models (DEQ) for video SCI, fusing data-driven regularization and stable convergence in a theoretically sound manner. Each equilibrium model implicitly learns a nonexpansive operator and analytically computes the fixed point, thus enabling unlimited iterative steps and infinite network depth with only a constant memory requirement in training and testing. Specifically, we demonstrate how DEQ can be applied to two existing models for video SCI reconstruction: recurrent neural networks (RNN) and Plug-and-Play (PnP) algorithms. On a variety of datasets and real data, both quantitative and qualitative evaluations of our results demonstrate the effectiveness and stability of our proposed method. The code and models will be released to the public.

show abstract

Snapshot Compressed Sensing: Performance Bounds and Algorithms

Cited by 95 publications

References 62 publications

Two-Stage is Enough: A Concise Deep Unfolding Reconstruction Network for Flexible Video Compressive Sensing

Two-Stage is Enough: A Concise Deep Unfolding Reconstruction Network for Flexible Video Compressive Sensing

Revisit Dictionary Learning for Video Compressive Sensing under the Plug-and-Play Framework

Deep Equilibrium Models for Video Snapshot Compressive Imaging

Contact Info

Product

Resources

About