Two-step domain adaptation for underwater image enhancement

Jiang, Qun; Zhang, Yunfeng; Bao, Fangxun; Zhao, Xiuyang; Zhang, Caiming; Лю, Пэйдэ

doi:10.1016/j.patcog.2021.108324

Cited by 88 publications

(21 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Noting that the goal of all of these works is to learn a domain-agnostic latent, in other words, to minimize the discrepancy of latent encoded from synthesis and real domains, however, the latent space is hard to manipulate and interpret, in contrast, we aim to separate image to content and style latent and distinguish extracted style latent from different domains, and further find a meaningful latent space that can be manipulated. Jiang et al [16] proposed a two-step domain adaptation framework without synthesis data, they use CycleGAN for style transfer to remove color cast, which is similar with synthesizing underwater image (clean → underwater) in [10] but in an opposite direction (underwater → clean), and remove hazy effect in second step.…”

Section: B Domain Adaptation For Underwater Imagementioning

confidence: 99%

“…The only restriction is images should at least contain underwater type degradation, which include color shift, low contrast, etc., but the degradation is not necessary to be realistic, since under the proposed framework, the style information would be separated from content and what we interested in are target domain (real-world underwater) style and clean high-quality image, and aim to build the pseudo real underwater image pair to learn the real-world underwater image enhancement. Noting that though the notion of performing domain adaptation via pseudo real-world image pair is similar with [15,16], we highlight two difference from [15,16]. First, the input of enhancement module is different, the prior works input the domain translation result, i.e.…”

Section: A Domain Adaptation and Image Enhancementmentioning

confidence: 99%

“…To solve the problem of lacking ground truth of real image, we generate pseudo real image pair, which is composed by syn2real underwater image and high-quality counterpart of synthesis image, to perform domain adaptation. Though the notion of generating pseudo real image pair to perform domain adaptation is common, we argue that disentangled image into content and style is more effective manner than encode domain-agnostic latent from different domain [12][13][14] or without latent space assumption [15,16], we will demonstrate the effectiveness in Section IV. Our…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Domain Adaptation for Underwater Image Enhancement via Content and Style Separation

Chen

Pei

2022

IEEE Access

View full text Add to dashboard Cite

Underwater image suffer from color cast, low contrast and hazy effect, which degraded the high-level vision application. Recent learning-based methods demonstrate astonishing performance on underwater image enhancement, however, most of these works use synthetic pair data for supervised learning and ignore the domain gap to real-world data. Although some work leverage transfer learning and domain adaptation to alleviate this problem, they target to minimize the latent discrepancy of synthesis and real-world data, and make the latent space hard to interpret and cannot manipulate. To solve this problem, we propose a domain adaptation framework for underwater image enhancement via content and style separation, we aim to separate encoded feature into content and style latent and distinguish style latent from different domains, and process domain adaptation and image enhancement in latent space. Our model provide a user interact interface to adjust different enhanced level for continuous change by latent manipulation. Experiment on various public real-world underwater benchmarks demonstrate that the proposed framework is capable to perform domain adaptation for underwater image enhancement and outperform various state-of-the-art underwater image enhancement algorithms in quantity and quality. The model and source code will be available at https://github.com/fordevoted/UIESS INDEX TERMS Underwater image enhancement, domain adaptation, content style disentanglement, real-world underwater images, deep learning.

show abstract

Section: B Domain Adaptation For Underwater Imagementioning

confidence: 99%

Section: A Domain Adaptation and Image Enhancementmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Domain Adaptation for Underwater Image Enhancement via Content and Style Separation

Chen

Pei

2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Noting that the goal of all of these works is to learn a domain-agnostic latent, in other words, to minimize the discrepancy of latent encoded from synthesis and real domains, in contrast, we aim to separate image to content and style latent and distinguish style latent from different sub-domains. Jiang et al [26] proposed a twostep domain adaptation framework without synthesis data, they use CycleGAN for style transfer to remove color cast, which is similar with synthesizing underwater image (clean → underwater) in [8] but in an opposite direction (underwater → clean), and remove hazy effect in second step.…”

Section: B Domain Adaptation For Underwater Imagementioning

confidence: 99%

Domain Adaptation for Underwater Image Enhancement via Content and Style Separation

Chen,

Pei

2022

Preprint

View full text Add to dashboard Cite

Underwater image suffer from color cast, low contrast and hazy effect due to light absorption, refraction and scattering, which degraded the high-level application, e.g, object detection and object tracking. Recent learning-based methods demonstrate astonishing performance on underwater image enhancement, however, most of these works use synthesis pair data for supervised learning and ignore the domain gap to real-world data. In this paper, we propose a domain adaptation framework for underwater image enhancement via content and style separation, we assume image could be disentangled to content and style latent, and image could be clustered to the sub-domain of associated style in latent space, the goal is to build up the mapping between underwater style latent and clean one. Different from prior works of domain adaptation for underwater image enhancement, which target to minimize the latent discrepancy of synthesis and real-world data, we aim to distinguish style latent from different sub-domains. To solve the problem of lacking pair real-world data, we leverage synthesis to real image-to-image translation to obtain pseudo real underwater image pairs for supervised learning, and enhancement can be achieved by input content and clean style latent into generator. Our model provide a user interact interface to adjust different enhanced level by latent manipulation. Experiment on various public real-world underwater benchmarks demonstrate that the proposed framework is capable to perform domain adaptation for underwater image enhancement and outperform various state-of-the-art underwater image enhancement algorithms in quantity and quality. The model and source code are available at https://github.com/fordevoted/UIESS

show abstract

“…However, these methods may not be e ective for all underwater images due to their diverse and complicated degradation types. Image restoration-based UIE methods [16][17][18][19][20][21][22][23][24][25][26][27][28][29][30][31] treat image quality amelioration as an inverse problem where the physical underwater optical model is utilized and diverse image priors are explored as constraints to estimate a global background light magnitude and a transmission matrix involved in the physical underwater optical model. However, one inevitable limitation of these prior-based restoration methods for UIE is that these priors are invalid to some specific underwater environments and/ or severe color casts.…”

Section: Introductionmentioning

confidence: 99%

Underwater Imaging Formation Model-Embedded Multiscale Deep Neural Network for Underwater Image Enhancement

et al. 2022

Mathematical Problems in Engineering

View full text Add to dashboard Cite

Underwater image enhancement (UIE) plays an essential role in improving the quality of raw images captured in an underwater environment. Existing UIE methods can be categorized into two types: handcraft-designed and deep learning-based methods. Generally, the handcraft-designed methods are more explainable due to the leverage of knowledge-based image priors, while the deep learning-based methods are usually criticized for their weak interpretability. In this study, we address this issue by integrating the merits of both handcraft-designed and deep learning-based methods. Specifically, a physical underwater imaging model-inspired deep CNN for UIE is designed. Instead of estimating a global background light magnitude and a transmission matrix separately in traditional image restoration-based UIE methods, we directly generate a single variable as the joint estimation of these two parameters within a deep CNN and directly recover the enhanced image as an output according to a reformulated physical underwater imaging model. The whole network is trained in an end-to-end manner and more importantly has good interpretability. The proposed method has been validated for the UIE task on a real-world underwater image dataset and the experimental results well demonstrate the superiority of our method over the existing ones for UIE.

show abstract

Two-step domain adaptation for underwater image enhancement

Cited by 88 publications

References 21 publications

Domain Adaptation for Underwater Image Enhancement via Content and Style Separation

Domain Adaptation for Underwater Image Enhancement via Content and Style Separation

Domain Adaptation for Underwater Image Enhancement via Content and Style Separation

Underwater Imaging Formation Model-Embedded Multiscale Deep Neural Network for Underwater Image Enhancement

Contact Info

Product

Resources

About