Video Error Concealment Using Deep Neural Networks

Neural Network-Based Error Concealment For VVC

“…These schemes address EC with analytical methods by exploiting spatio-temporal information available in the decoder to construct the lost portion of the video. In [13], a deep neuronal network was trained to emulate EC for a single lost slice assuming a frame is divided in multiple slices. Its performance was not measured within any coding standard.…”

Image Error Concealment Based on Deep Neural Network

“…The basic idea behind EC is to predict the missing pixels by using the correctly received ones in the current frame or adjacent frames based on the spatial or temporal correlations. According to which kind of correlation is utilized, EC methods can be classified into three categories: spatial EC (SEC) [2][3][4][5][6][7][8][9][10][11][12][13][14][15], temporal EC (TEC) [16][17][18][19][20][21][22], or spatial-temporal EC (STEC) [23][24][25][26]. In the case that the neighbor frames are not available, the SEC methods only use the information extracted from the neighboring pixels of the missing ones in the current frame.…”

“…Neural network, as a powerful model, has been proved to be effective in EC tasks [15,20,21]. Shao and Chen [20] exploited a general regression neural network (GRNN) to estimate the motion vectors of the corrupted macro-blocks (MBs).…”

“…Then, the corrupted MBs were reconstructed through the corresponding motion vectors, which were estimated by the GRNN. In [21], deep neural networks were used to predict the optical flow for video error concealment. They designed two parallel networks to separately process the horizontal and vertical motion fields of the optical flows of the previous frame.…”

“…Unlike [20,21], we reconstructed the missing pixels through only utilizing the information from the current frame in this paper. We propose a novel EC technique to reconstruct the missing pixels via training a deep neural network.…”