JsrNet: A Joint Sampling–Reconstruction Framework for Distributed Compressive Video Sensing

Abstract

Huge video data has posed great challenges on computing power and storage space, triggering the emergence of distributed compressive video sensing (DCVS). Hardware-friendly characteristics of this technique have consolidated its position as one of the most powerful architectures in source-limited scenarios, namely, wireless video sensor networks (WVSNs). Recently, deep convolutional neural networks (DCNNs) are successfully applied in DCVS because traditional optimization-based methods are computationally elaborate and hard to meet the requirements of real-time applications. In this paper, we propose a joint sampling–reconstruction framework for DCVS, named “JsrNet”. JsrNet utilizes the whole group of frames as the reference to reconstruct each frame, regardless of key frames and non-key frames, while the existing frameworks only utilize key frames as the reference to reconstruct non-key frames. Moreover, different from the existing frameworks which only focus on exploiting complementary information between frames in joint reconstruction, JsrNet also applies this conception in joint sampling by adopting learnable convolutions to sample multiple frames jointly and simultaneously in an encoder. JsrNet fully exploits spatial–temporal correlation in both sampling and reconstruction, and achieves a competitive performance in both the quality of reconstruction and computational complexity, making it a promising candidate in source-limited, real-time scenarios.

Imrnet: An Iterative Motion Compensation and Residual Reconstruction Network for Video Compressed Sensing

“…For the fairness of experimental comparison, as in [12,13,14,15], we use the UCF-101 dataset for model training and randomly divide all video sequences into 80% as the training set, 10% as the validation set and 10% as the test set. Each frame is centrally cropped to a size of 160 × 160 and only the lumimance component in the yCbCr color space is retained.…”

“…In [12,15,14], GOP size (T) is 10, the sampling rate of key frames is 0.2, and the sampling rate of non-key frames is 0.037, 0.018 and 0.009. In [13], the GOP size (T) is 4, the sampling rate of key frames is 0.25, and the sampling rate of non-key frames is 0.1, 0.04, and 0.01. We respectively used the same parameter settings for comparative experiments, and the experimental results are shown in Table 2 and Table 3.…”

“…Current deep learning methods have achieved better performance than traditional methods in the field of image compressed sensing [9,10,11] and video compressed sensing [12,13,14,15] while being significantly faster at reconstruction time. Through stacking some naive inception modules and convolutional layers, JsrNet [13] directly learns the nonlinear mapping of image groups from the observation domain to pixel domain. CSVideoNet [12] obtains motion compensation of non-key frames through LSTM network.…”

High-Resolution Representations Network for Single Image Dehazing

“…In recent years, with the development of artificial intelligence, computer vision has been applied to all aspects of our lives [ 1 , 2 ]. These high-level image processing tasks have high requirements for the quality of the input image.…”

Deep learning approaches for challenging species and gender identification of mosquito vectors

“…Deep learning algorithms such as convolutional neural networks (CNNs) have been noted to be effective in several object detection and image recognition applications. For instance, CNNs have been extensively applied to detect cyclists on roads and pests in agriculture, as well as in many medical applications 18 – 22 . These examples indicate that CNNs represent a promising tool to be applied in the entomological field to detect the species and gender of mosquito vectors.…”