Distorted underwater image reconstruction for an autonomous underwater vehicle based on a self-attention generative adversarial network

Abstract

Imaging through the wavy air–water surface suffers from severe geometric distortions, which are caused by the light refraction effect that affects the normal operations of underwater exploration equipment such as the autonomous underwater vehicle (AUV). In this paper, we propose a deep learning-based framework, namely the self-attention generative adversarial network (SAGAN), to remove the geometric distortions and restore the distorted image captured through the water–air surface. First, a K-means-based image pre-selection method is employed to acquire a less distorted image that preserves much useful information from an image sequence. Second, an improved generative adversarial network (GAN) is trained to translate the distorted image into the non-distorted image. During this process, the attention mechanism and the weighted training objective are adopted in our GAN framework to get the high-quality restored results of distorted underwater images. The network is able to restore the colors and fine details in the distorted images by combining the three objective losses, i.e., the content loss, the adversarial loss, and the perceptual loss. Experimental results show that our proposed method outperforms other state-of-the-art methods on the validation set and our sea trial set.

Underwater Motion Deblurring Based on Cascaded Attention Mechanism

“…An image translation GAN framework is notoriously hard to train. In previous works [26] and [36], a weighted loss function showed satisfactory performance in training a complex GAN mapping framework.…”

“…θ is the parameter of the deep CNN. Existing deep learning frameworks, especially GAN [36][37][38], achieved great success in the field of image translation tasks. There are two competing networks in a standard GAN, namely the generator network and the discriminative network.…”

“…It is a three-term loss function, which consists of the content loss L con , the adversarial loss L adv , and the perceptual loss L per . Among them, the content loss L con can yield over smoothened pixel-space outputs [36,38]. As the underwater scenes are usually dark, and the camera or the object is in a motion condition, the captured underwater images suffer from different degrees of blur.…”

“…However, L con alone cannot generate satisfactory resultant images, the resultant images are still blurry and usually lack high frequency details [20]. Hence, relativistic average least squares GAN [36] (RaLSGAN) objective loss (the adversarial loss L adv ), is used to further to improve the high frequency details in the images. It has proven in [17] that L adv can allow the network to learn sharper edges and more detailed textures by estimating the probability that the original image is more realistic or not than the blurry image reconstructed by the generator.…”