Abstract:The paper proposes a novel post-filtering method based on convolutional neural networks (CNNs) for quality enhancement of RGB/grayscale images and video sequences. The lossy images are encoded using common image codecs, such as JPEG and JPEG2000. The video sequences are encoded using previous and ongoing video coding standards, high-efficiency video coding (HEVC) and versatile video coding (VVC), respectively. A novel deep neural network architecture is proposed to estimate fine refinement details for full-, h… Show more
“…The architecture of deep learning-based restoration methods are commonly made up of three layers: feature extraction, feature processing, and reconstruction. To realize the reconstruction layer, two approaches based on the output of the predicted content can be used: by predicting the whole restored image [6,[9][10][11]13,14] or by predicting the residual information for compensating the artifacts [5,7,12,[15][16][17]19,20]. Because the amount of information required to predict residual information is less, the residual information prediction is less difficult and the accuracy can be improved.…”
Section: Single Image Restoration Methodsmentioning
confidence: 99%
“…It proves the importance of restoring color artifacts, which is why we developed our method for processing both the luminance and chroma. Table 4 compares the results of our method to those of other studies [10,[12][13][14][15][16][17] tested in the LIVE1 dataset. Since no other existing work provides data for the chroma channel, we will only compare our chroma improvement to the chroma quality of JPEG-compressed images.…”
Section: Experiments Of Single-image Restoration Methodsmentioning
confidence: 99%
“…Several researchers [10][11][12] designed shallow models to retrieve spatial information and generate restoration results. Unfortunately, the restoration results were unsatisfactory, and thus, a deeper network architecture was adopted [13][14][15][16][17]. Cavigelli et al [13] adjusted the filter stride in the network to process the feature maps at four different scales to accommodate different object sizes.…”
Video compression algorithms are commonly used to reduce the number of bits required to represent a video with a high compression ratio. However, this can result in the loss of content details and visual artifacts that affect the overall quality of the video. We propose a learning-based restoration method to address this issue, which can handle varying degrees of compression artifacts with a single model by predicting the difference between the original and compressed video frames to restore video quality. To achieve this, we adopted a recursive neural network model with dilated convolution, which increases the receptive field of the model while keeping the number of parameters low, making it suitable for deployment on a variety of hardware devices. We also designed a temporal fusion module and integrated the color channels into the objective function. This enables the model to analyze temporal correlation and repair chromaticity artifacts. Despite handling color channels, and unlike other methods that have to train a different model for each quantization parameter (QP), the number of parameters in our lightweight model is kept to only about 269 k, requiring only about one-twelfth of the parameters used by other methods. Our model applied to the HEVC test model (HM) improves the compressed video quality by an average of 0.18 dB of BD-PSNR and −5.06% of BD-BR.
“…The architecture of deep learning-based restoration methods are commonly made up of three layers: feature extraction, feature processing, and reconstruction. To realize the reconstruction layer, two approaches based on the output of the predicted content can be used: by predicting the whole restored image [6,[9][10][11]13,14] or by predicting the residual information for compensating the artifacts [5,7,12,[15][16][17]19,20]. Because the amount of information required to predict residual information is less, the residual information prediction is less difficult and the accuracy can be improved.…”
Section: Single Image Restoration Methodsmentioning
confidence: 99%
“…It proves the importance of restoring color artifacts, which is why we developed our method for processing both the luminance and chroma. Table 4 compares the results of our method to those of other studies [10,[12][13][14][15][16][17] tested in the LIVE1 dataset. Since no other existing work provides data for the chroma channel, we will only compare our chroma improvement to the chroma quality of JPEG-compressed images.…”
Section: Experiments Of Single-image Restoration Methodsmentioning
confidence: 99%
“…Several researchers [10][11][12] designed shallow models to retrieve spatial information and generate restoration results. Unfortunately, the restoration results were unsatisfactory, and thus, a deeper network architecture was adopted [13][14][15][16][17]. Cavigelli et al [13] adjusted the filter stride in the network to process the feature maps at four different scales to accommodate different object sizes.…”
Video compression algorithms are commonly used to reduce the number of bits required to represent a video with a high compression ratio. However, this can result in the loss of content details and visual artifacts that affect the overall quality of the video. We propose a learning-based restoration method to address this issue, which can handle varying degrees of compression artifacts with a single model by predicting the difference between the original and compressed video frames to restore video quality. To achieve this, we adopted a recursive neural network model with dilated convolution, which increases the receptive field of the model while keeping the number of parameters low, making it suitable for deployment on a variety of hardware devices. We also designed a temporal fusion module and integrated the color channels into the objective function. This enables the model to analyze temporal correlation and repair chromaticity artifacts. Despite handling color channels, and unlike other methods that have to train a different model for each quantization parameter (QP), the number of parameters in our lightweight model is kept to only about 269 k, requiring only about one-twelfth of the parameters used by other methods. Our model applied to the HEVC test model (HM) improves the compressed video quality by an average of 0.18 dB of BD-PSNR and −5.06% of BD-BR.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.