Recurrent Neural Networks for Compressive Video Reconstruction

Abstract

Single-pixel imaging allows low cost cameras to be built for imaging modalities where a conventional camera would either be too expensive or too cumbersome. This is very attractive for biomedical imaging applications based on hyperspectral measurements, such as image-guided surgery, which requires the full spectrum of fluorescence. A single-pixel camera essentially measures the inner product of the scene and a set of patterns. An inverse problem has to be solved to recover the original image from the raw measurement. The challenge in single-pixel imaging is to reconstruct the video sequence in real time from under-sampled data. Previous approaches have focused on the reconstruction of each frame independently, which fails to exploit the natural temporal redundancy in a video sequence. In this study, we propose a fast deep-learning reconstructor that exploits the spatio-temporal features in a video. In particular, we consider convolutional gated recurrent units that have low memory requirements. Our simulation shows than the proposed recurrent network improves the reconstruction quality compared to static approaches that reconstruct the video frames independently.

Towards Low-Cost Hyperspectral Single-Pixel Imaging for Plant Phenotyping

“…Figure 2b shows a 101 × 101 pixels measurement obtained over 4 h. The acquisition time can be reduced by three complementary optimizations, namely algorithmic, electronical, and optical optimizations. The algorithmic improvement of compressive sensing acquisitions is an active field of studies [15,23,24]. Fourier patterns are used in the study for the sake of simplicity but could be benchmarked against the basis of other patterns and reconstruction strategies.…”

Optimal Reconstruction of Single-Pixel Images through Feature Feedback Mechanism and Attention

“…Recent research has shown that feedback mechanisms have been applied to a variety of computer vision tasks [37][38][39][40]. For the SPI task, Antonio et al used an RNN network with temporal memory to control the delivery and loss of features [41]. Ikuo et al improved the quality of the reconstructed images by chunking the measurement data input and accumulating and updating them in an RNN [42].…”

An Overview on Deep Learning Techniques for Video Compressive Sensing

“…Recently, deep neural networks have been used successfully in signal pixel imaging reconstruction problems. In [66], A. l. Mur et al have exploited the spatio-temporal features of video and proposed a Convolutional Gated Recurrent Units (ConvGRU) based algorithm to reconstruct video frames already captured by a single pixel camera. N. Ducros et al [67] defined a generic convolutional network to recover the original video.…”