WaterGAN: Unsupervised Generative Network to Enable Real-time Color Correction of Monocular Underwater Images

Abstract

Abstract-This paper reports on WaterGAN, a generative adversarial network (GAN) for generating realistic underwater images from in-air image and depth pairings in an unsupervised pipeline used for color correction of monocular underwater images. Cameras onboard autonomous and remotely operated vehicles can capture high resolution images to map the seafloor; however, underwater image formation is subject to the complex process of light propagation through the water column. The raw images retrieved are characteristically different than images taken in air due to effects such as absorption and scattering, which cause attenuation of light at different rates for different wavelengths. While this physical process is well described theoretically, the model depends on many parameters intrinsic to the water column as well as the structure of the scene. These factors make recovery of these parameters difficult without simplifying assumptions or field calibration; hence, restoration of underwater images is a non-trivial problem. Deep learning has demonstrated great success in modeling complex nonlinear systems but requires a large amount of training data, which is difficult to compile in deep sea environments. Using WaterGAN, we generate a large training dataset of corresponding depth, in-air color images, and realistic underwater images. This data serves as input to a two-stage network for color correction of monocular underwater images. Our proposed pipeline is validated with testing on real data collected from both a pure water test tank and from underwater surveys collected in the field. Source code, sample datasets, and pretrained models are made publicly available.

Siamese-GAN: Learning Invariant Representations for Aerial Vehicle Image Categorization

“…For instance, Deep Convolutional GANs (DCGANs) [38] were designed to allow the network to generate data with similar internal structure as training data, improving the quality of the generated images, and Conditional GANs [39] add an additional conditioning variable to both the generator and the discriminator. Based on the previous architectures the concept of GANs has been adopted to solve many computer visions related tasks such as image generation [40,41], image super-resolution [42], unsupervised learning [43], semi-supervised learning [44], and image painting and colorization [45,46].…”

Emerging From Water: Underwater Image Color Correction Based on Weakly Supervised Color Transfer

“…Compared with traditional methods based on statistical priors, we bridge the gap between the color of underwater image and that of air image by learning their cross domain relations. Recently, a semi-supervised learning model for underwater image color correction, namely WaterGAN, has been proposed [14]. Unlike this work, our model is no need for a large annotated dataset of images pairs.…”

MLFcGAN: Multilevel Feature Fusion-Based Conditional GAN for Underwater Image Color Correction

“…In [10], the convolutional neural network (CNN) was trained to approximate the underwater image restoration function given the synthesized paired underwater images. Li et al in [11] proposed a two-stage CNN for depth estimation and color restoration. Recently, the generative adversarial networks have achieved huge success on many tasks such as super-resolution [12] and image synthesis and translation [13].…”

“…Generally speaking, such CNN-based methods outperform aforementioned model-based methods. However, methods in [10], [11] and [14] only leverage low-level local features with relatively shallow networks and such low-level features captured with limited receptive field can hardly encode the high-level semantic knowledge, resulting in noisy and imperfect color restoration results. To overcome the limitations of available CNN-based methods, we propose to exploit high-level features in the cGAN framework for underwater image color correction.…”