RFA-Net: Residual feature attention network for fine-grained image inpainting

Chen, Min; Zang, Shengrui; Ai, Zhenhua; Chi, J.Y.; Yang, Guowei; Chen, Chenglizhao; Teng, Yunlong

doi:10.1016/j.engappai.2022.105814

Cited by 13 publications

(6 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Vo et al proposed a divide-and-conquer algorithm to address the sub-problems of degraded face reconstruction and classification by effectively applying multiple deep convolutional neural networks [31]. Another advanced texture-aware network named RFA-Net [32] employed a non-pooling residual CNN with three novel modules for finer image inpainting under the supervision of hybrid loss optimization, focusing on the semantic and texture details of the inpainting. Later, a generative adversarial network (GAN) was utilized to reconstruct faces by cooperating with a pre-trained convolutional neural network (CNN) while sustaining identity-variance features [33].…”

Section: Deep-learning-based Methodsmentioning

confidence: 99%

Adversarially Learning Occlusions by Backpropagation for Face Recognition

Zhao,

Qin,

Zhang

2023

Sensors

View full text Add to dashboard Cite

With the accomplishment of deep neural networks, face recognition methods have achieved great success in research and are now being applied at a human level. However, existing face recognition models fail to achieve state-of-the-art performance in recognizing occluded face images, which are common scenarios captured in the real world. One of the potential reasons for this is the lack of large-scale training datasets, requiring labour-intensive and costly labelling of the occlusions. To resolve these issues, we propose an Adversarially Learning Occlusions by Backpropagation (ALOB) model, a simple yet powerful double-network framework used to mitigate manual labelling by contrastively learning the corrupted features against personal identity labels, thereby maximizing the loss. To investigate the performance of the proposed method, we compared our model to the existing state-of-the-art methods, which function under the supervision of occlusion learning, in various experiments. Extensive experimentation on LFW, AR, MFR2, and other synthetic masked or occluded datasets confirmed the effectiveness of the proposed model in occluded face recognition by sustaining better results in terms of masked face recognition and general face recognition. For the AR datasets, the ALOB model outperformed other advanced methods by obtaining a 100% recognition rate for images with sunglasses (protocols 1 and 2). We also achieved the highest accuracies of 94.87%, 92.05%, 78.93%, and 71.57% TAR@FAR = 1 × 10−3 in LFW-OCC-2.0 and LFW-OCC-3.0, respectively. Furthermore, the proposed method generalizes well in terms of FR and MFR, yielding superior results in three datasets, LFW, LFW-Masked, and MFR2, and producing accuracies of 98.77%, 97.62%, and 93.76%, respectively.

show abstract

Section: Deep-learning-based Methodsmentioning

confidence: 99%

Adversarially Learning Occlusions by Backpropagation for Face Recognition

Zhao,

Qin,

Zhang

2023

Sensors

View full text Add to dashboard Cite

show abstract

“…This helps the model to better understand the details and structures in the image, resulting in improved restoration quality and accuracy. In addition, the utilization of efficient channel attention [38] and pixel attention [39] enables the model to selectively focus on important channels and pixels, thus reducing unnecessary computations and parametric quantities. The residual structure and skip connections help to avoid gradient explosion and network convergence difficulties.…”

Section: Multi-scale Fusion Attention Modulementioning

confidence: 99%

Image Inpainting Technique Incorporating Edge Prior and Attention Mechanism

Bai,

Fan,

Zhao

et al. 2024

CMC

View full text Add to dashboard Cite

Recently, deep learning-based image inpainting methods have made great strides in reconstructing damaged regions. However, these methods often struggle to produce satisfactory results when dealing with missing images with large holes, leading to distortions in the structure and blurring of textures. To address these problems, we combine the advantages of transformers and convolutions to propose an image inpainting method that incorporates edge priors and attention mechanisms. The proposed method aims to improve the results of inpainting large holes in images by enhancing the accuracy of structure restoration and the ability to recover texture details. This method divides the inpainting task into two phases: edge prediction and image inpainting. Specifically, in the edge prediction phase, a transformer architecture is designed to combine axial attention with standard self-attention. This design enhances the extraction capability of global structural features and location awareness. It also balances the complexity of self-attention operations, resulting in accurate prediction of the edge structure in the defective region. In the image inpainting phase, a multi-scale fusion attention module is introduced. This module makes full use of multi-level distant features and enhances local pixel continuity, thereby significantly improving the quality of image inpainting. To evaluate the performance of our method, comparative experiments are conducted on several datasets, including CelebA, Places2, and Facade. Quantitative experiments show that our method outperforms the other mainstream methods. Specifically, it improves Peak Signal-to-Noise Ratio (PSNR) and Structure Similarity Index Measure (SSIM) by 1.141∼3.234 db and 0.083∼0.235, respectively. Moreover, it reduces Learning Perceptual Image Patch Similarity (LPIPS) and Mean Absolute Error (MAE) by 0.0347∼0.1753 and 0.0104∼0.0402, respectively. Qualitative experiments reveal that our method excels at reconstructing images with complete structural information and clear texture details. Furthermore, our model exhibits impressive performance in terms of the number of parameters, memory cost, and testing time.

show abstract

“…GAN has attracted increasing attention in image inpainting due to its rapid development and good generative performance [11,12,[21][22][23][24]. The method proposed by Yan et al [25] can yield outstanding results when solving rectangles or small holes.…”

Section: Related Workmentioning

confidence: 99%

“…GAN has attracted increasing attention in image inpainting due to its rapid development and good generative performance [11, 12, 21–24]. The method proposed by Yan et al.…”

Section: Related Workmentioning

confidence: 99%

Image inpainting network based on multi‐level attention mechanism

Xiang,

Min,

Wei

et al. 2023

IET Image Processing

View full text Add to dashboard Cite

Image inpainting networks based on deep learning techniques have been widely used in many important fields. However, most inpainting networks fail to generate desirable repaired images. This may be due to their failure to extract effective features and accurately assign high weights to the undamaged regions. To alleviate these problems, an image inpainting network based on gated convolution and multi‐level attention mechanism (IIN‐GCMAM) is proposed in this paper. This network follows encoder–decoder architecture, consisting of the gated convolution encoder (GC‐encoder) and the multi‐level attention mechanism decoder (MAM‐decoder). The GC‐encoder weighs the extracted features with gated convolutions, which reduces the interference caused by the damaged regions. The multi‐level attention mechanism employed in the MAM‐decoder uses multi‐scale feature maps spatially and channel‐wise to improve the consistency in global structure and the fineness of repaired results. Extensive experiments are conducted on the common datasets, Paris StreetView and CelebA. Experimental results indicate that the proposed IIN‐GCMAM can achieve a good performance on the common evaluation metrics and visual effects. It can achieve 0.0408, 0.720, and 22.27 in MAE, SSIM, and PSNR at the mask ratio of 50%–60%, respectively.

show abstract

RFA-Net: Residual feature attention network for fine-grained image inpainting

Cited by 13 publications

References 17 publications

Adversarially Learning Occlusions by Backpropagation for Face Recognition

Adversarially Learning Occlusions by Backpropagation for Face Recognition

Image Inpainting Technique Incorporating Edge Prior and Attention Mechanism

Image inpainting network based on multi‐level attention mechanism

Contact Info

Product

Resources

About