GAN-Based Image Colorization for Self-Supervised Visual Feature Learning

Abstract

Large-scale labeled datasets are generally necessary for successfully training a deep neural network in the computer vision domain. In order to avoid the costly and tedious work of manually annotating image datasets, self-supervised learning methods have been proposed to learn general visual features automatically. In this paper, we first focus on image colorization with generative adversarial networks (GANs) because of their ability to generate the most realistic colorization results. Then, via transfer learning, we use this as a proxy task for visual understanding. Particularly, we propose to use conditional GANs (cGANs) for image colorization and transfer the gained knowledge to two other downstream tasks, namely, multilabel image classification and semantic segmentation. This is the first time that GANs have been used for self-supervised feature learning through image colorization. Through extensive experiments with the COCO and Pascal datasets, we show an increase of 5% for the classification task and 2.5% for the segmentation task. This demonstrates that image colorization with conditional GANs can boost other downstream tasks’ performance without the need for manual annotation.

A Novel Knowledge Distillation Method for Self-Supervised Hyperspectral Image Classification

“…Since self-supervised learning does not require the data itself to have labeled information, it has a wide range of applications in various fields [39][40][41][42][43]. The main auxiliary tasks in the image domain include Jigsaw Puzzles [44], Image Colorization [45], image rotation [46], image restoration [47], image fusion [48] and so on. In the case of image rotation, for example, the unlabeled image can be rotated by four angles and given labels, and the rotated image and the original image are fed into the network to predict the rotation angle.…”

Self-supervised Learning: A Succinct Review

“…It is a pretext for learning visual features. Treneska et al [ 19 ] used an image colorization model based on a Generative Adversarial Network (GAN). The GAN model can produce the most realistic results.…”

Colorizing the Past: Deep Learning for the Automatic Colorization of Historical Aerial Images

“…GANs-based colorization frameworks are progressively replacing more straightforward CNNs methods, despite their greater complexity [ 54 , 55 , 56 , 57 , 58 , 59 , 60 , 61 , 62 ]. These implementations adopt several learning strategies and network architectures, sometimes coupling the adversarial learning colorization with further perceptual or semantic information [ 59 , 60 ], or proposing a flexible framework for addressing several image-to-image translation problems [ 56 ].…”