An Efficient Convolutional Neural Network Model Combined with Attention Mechanism for Inverse Halftoning

Abstract

Inverse halftoning acting as a special image restoration problem is an ill-posed problem. Although it has been studied in the last several decades, the existing solutions can’t restore fine details and texture accurately from halftone images. Recently, the attention mechanism has shown its powerful effects in many fields, such as image processing, pattern recognition and computer vision. However, it has not yet been used in inverse halftoning. To better solve the problem of detail restoration of inverse halftoning, this paper proposes a simple yet effective deep learning model combined with the attention mechanism, which can better guide the network to remove noise dot-patterns and restore image details, and improve the network adaptation ability. The whole model is designed in an end-to-end manner, including feature extraction stage and reconstruction stage. In the feature extraction stage, halftone image features are extracted and halftone noises are removed. The reconstruction stage is employed to restore continuous-tone images by fusing the feature information extracted in the first stage and the output of the residual channel attention block. In this stage, the attention block is firstly introduced to the field of inverse halftoning, which can make the network focus on informative features and further enhance the discriminative ability of the network. In addition, a multi-stage loss function is proposed to accelerate the network optimization, which is conducive to better reconstruction of the global image. To demonstrate the generalization performance of the network for different types of halftone images, the experiment results confirm that the network can restore six different types of halftone image well. Furthermore, experimental results show that our method outperforms the state-of-the-art methods, especially in the restoration of details and textures.

An inverse halftoning method based on supervised deep convolutional neural network

“…In addition, the subnetworks of these methods mainly apply Sobel operator and the residual blocks or one of them, so the gradient information can be better recognized. Recently, the attention mechanism has also been applied into the inverse halftoning methods, such as attention mechanism for inverse halftoning (called AM‐IH method) [33], which has achieved better results. The image details in inverse halftone images restored by these methods are enhanced, but the halftone noise dots cannot be eliminated completely.…”

Efficient Shot Detector: Lightweight Network Based on Deep Learning Using Feature Pyramid

“…Then, the element summing is performed for each pixel. A low-cost convolution for the fused features was used as shown in Equation (3). However, all input features are treated equally when a refinement process is added between layers by simply convolution.…”

“…The rapid advancements and current level of computational power of deep learning based methods can be used in several applications, including autonomous driving systems [1], air traffic control [2], and image restoration [3], with high accuracy, which exhibit their capacity to replace the existing and traditional systems. However, high latency networks and traffic problems occur when processing an infinite amount of data using a graphics processing unit (GPU) based cloud system.…”

Efficient Iterative Regularization Method for Total Variation-Based Image Restoration

“…If we ignore the specific form of ϕ(x), superior denoisers such as non-local means filter [21], block-matching and 3D filtering (BM3D) [22], bilateral filter [23], and adversarial Gaussian denoiser [24], can be adopted for solving this denoising subproblem. Moreover, with the rapid development of deep learning technology in image denoising, super-resolution reconstruction, object detection and control [25,26], deep learning methods [27][28][29][30] using clean-noisy image pairs have been widely exploited in the design of denoisers. Multi-layer perceptron was adopted for image restoration in [27] while various convolutional neural network (CNN) and generative adversarial networks methods have been used to design specific denoisers [28][29][30].…”

“…Moreover, with the rapid development of deep learning technology in image denoising, super-resolution reconstruction, object detection and control [25,26], deep learning methods [27][28][29][30] using clean-noisy image pairs have been widely exploited in the design of denoisers. Multi-layer perceptron was adopted for image restoration in [27] while various convolutional neural network (CNN) and generative adversarial networks methods have been used to design specific denoisers [28][29][30]. It is well known that neural network methods are limited in computing speed and high requirements of hardware, with no universal adaptation for the applications that require simplicity and rapidity.…”