Convolutional Neural Networks for Noniterative Reconstruction of Compressively Sensed Images

Lohit, Suhas; Kulkarni, Kuldeep; Kerviche, Ronan; Turaga, Pavan; Ashok, Amit

doi:10.1109/tci.2018.2846413

Cited by 110 publications

(78 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An emerging application for deep learning is to learn how to both optimally collect and process data for some end goal, rather than just process data [41][42][43][44][45]. With this "physical preprocessing" procedure for deep learning, data acquisition time or hardware requirements can be reduced.…”

Section: Introductionmentioning

confidence: 99%

Optimal physical preprocessing for example-based super-resolution

Robey¹,

Ganapati²

2018

Opt. Express

View full text Add to dashboard Cite

In example-based super-resolution, the function relating low-resolution images to their high-resolution counterparts is learned from a given dataset. This data-driven approach to solving the inverse problem of increasing image resolution has been implemented with deep learning algorithms. In this work, we explore modifying the imaging hardware in order to collect more informative low-resolution images for better ultimate high-resolution image reconstruction. We show that this "physical preprocessing" allows for improved image reconstruction with deep learning in Fourier ptychographic microscopy.Fourier ptychographic microscopy is a technique allowing for both high resolution and high field-of-view at the cost of temporal resolution. In Fourier ptychographic microscopy, variable illumination patterns are used to collect multiple low-resolution images. These low-resolution images are then computationally combined to create an image with resolution exceeding that of any single image from the microscope. We use deep learning to jointly optimize the illumination pattern with the post-processing reconstruction algorithm for a given sample type, allowing for single-shot imaging with both high resolution and high field-of-view. We demonstrate, with simulated data, that the joint optimization yields improved image reconstruction as compared with sole optimization of the post-processing reconstruction algorithm.

show abstract

Section: Introductionmentioning

confidence: 99%

Optimal physical preprocessing for example-based super-resolution

Robey¹,

Ganapati²

2018

Opt. Express

View full text Add to dashboard Cite

show abstract

“…Based on ReconNet, DR 2 -Net [8] obtains an improved result by learning the residual information between low resolution images and ground truth images. Adp-Rec [11], DeepCodec [12] and new ReconNet [9] jointly train the measurement and the reconstruction parts. They acquire excellent performance.…”

Section: Cnn-based Csmentioning

confidence: 99%

“…Compressive sensing (CS) theory shows that signal can be reconstructed under an extremely low sample rate because of its sparse structure. In conventional CS problem, signal is measured with Gaussian matrix in blocks and then recovered by optimization algorithms [ Recently, CNN-based methods [7] [8] [9] [10] are proposed for CS problem. The network can adaptively learn a transform from measurements to reconstruction images by minimizing error between the original and the reconstructed images in large dataset.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Full Image Recover for Block-Based Compressive Sensing

Xie

Wang

et al. 2018

2018 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

Recent years, compressive sensing (CS) has improved greatly for the application of deep learning technology. For convenience, the input image is usually measured and reconstructed block by block. This usually causes block effect in reconstructed images. In this paper, we present a novel CNN-based network to solve this problem. In measurement part, the input image is adaptively measured block by block to acquire a group of measurements. While in reconstruction part, all the measurements from one image are used to reconstruct the full image at the same time. Different from previous methods recovering block by block, the structure information destroyed in measurement part is recovered in our framework. Block effect is removed accordingly. We train the proposed framework by mean square error (MSE) loss function. Experiments show that there is no block effect at all in the proposed method. And our results outperform 1.8 dB compared with existing methods.Index Terms-Compressive sensing, convolutional neural network (CNN), block-wise, full image recover

show abstract

“…(1) Block effect occurs in Fig 3 (b) and (c) by block-wise methods such as ReconNet [18] and Adp-Rec [37]. Based on the standard ReconNet [18], the improved ReconNet [22] adds several tricks such as adaptive measurement and adversarial loss. Its performance is even lower than Adp-Rec [37].…”

Section: Experiments With Analysismentioning

confidence: 99%

Perceptual Compressive Sensing

Xie

Wang

et al. 2018

Pattern Recognition and Computer Vision

View full text Add to dashboard Cite

Compressive sensing (CS) works to acquire measurements at sub-Nyquist rate and recover the scene images. Existing CS methods always recover the scene images in pixel level. This causes the smoothness of recovered images and lack of structure information, especially at a low measurement rate. To overcome this drawback, in this paper, we propose perceptual CS to obtain high-level structured recovery. Our task no longer focuses on pixel level. Instead, we work to make a better visual effect. In detail, we employ perceptual loss, defined on feature level, to enhance the structure information of the recovered images. Experiments show that our method achieves better visual results with stronger structure information than existing CS methods at the same measurement rate. 2 J. Du et al.the CS measurements can be recovered real-time. [18] [25] learns the recovery network by the training data. Adp-Rec [37] jointly train the coder-decoder and brings significant improvement on reconstruction quality. Fully convolutional measurement network (FCMN) [36] firstly measures and recovers full images. However, all the above methods focus on pixel level, and ignore the high-level structure information. This makes the reconstructed results look smooth and have unsatisfactory visual effect. To overcome the drawback, we consider to add high-level perceptual information to CS. So the question is, how to add high-level perceptual information on the low-level CS task.Recently, perceptual loss [17] has been used in many reconstruction tasks, such as style transfer [17] [5]. They are a combination of low-level detailed information and high level semantic information. Perceptual loss is a widely used way to achieve these goals. It is because perceptual loss is defined in feature space, which can convert the ability of catching high-level structure information to recovery network. Thus, the recovered images will contain rich structure information. Inspired by the above applications, we propose perceptual CS, which focuses more on sensing and recovering structure information. Here perceptual loss is employed on CS framework. We use FCMN [36] as base network to measure and recover scene images, and adopt perceptual loss to train it. We surprisingly find that this framework is capable of capturing and recovering the structure information, especially at extremely low measurement rate, where the measurements can merely contain very limited amount of information.The contribution of this paper is that, we propose perceptual CS, which can measure and recover the structure information of scene images. It should be pointed out that, only one deconvolutional layer and one Res-block are used in our proposed framework. This is just an illustration. One can employ a deeper network architecture if necessary.Moreover, perceptual CS indicates an universal architecture. One can change the loss network using pre-trained or dynamic feature extractors for more specific tasks. In this paper, we use VGG [32] as an example. Our code will be available on github 1 fo...

show abstract

Convolutional Neural Networks for Noniterative Reconstruction of Compressively Sensed Images

Cited by 110 publications

References 57 publications

Optimal physical preprocessing for example-based super-resolution

Optimal physical preprocessing for example-based super-resolution

Full Image Recover for Block-Based Compressive Sensing

Perceptual Compressive Sensing

Contact Info

Product

Resources

About