Learning a Deep ConvNet for Multi-Label Classification With Partial Labels

Abstract

Deep ConvNets have shown great performance for single-label image classification (e.g. ImageNet), but it is necessary to move beyond the single-label classification task because pictures of everyday life are inherently multilabel. Multi-label classification is a more difficult task than single-label classification because both the input images and output label spaces are more complex. Furthermore, collecting clean multi-label annotations is more difficult to scale-up than single-label annotations. To reduce the annotation cost, we propose to train a model with partial labels i.e. only some labels are known per image. We first empirically compare different labeling strategies to show the potential for using partial labels on multi-label datasets. Then to learn with partial labels, we introduce a new classification loss that exploits the proportion of known labels per example. Our approach allows the use of the same training settings as when learning with all the annotations. We further explore several curriculum learning based strategies to predict missing labels. Experiments are performed on three large-scale multi-label datasets: MS COCO, NUS-WIDE and Open Images.

Confidence-based Weighted Loss for Multi-label Classification with Missing Labels

“…Weighted loss functions is a common approach for different problems, e.g. to solve class imbalance [32], to focus on samples that are harder to predict [20], or to solve a similar problem of partial labels [11]. However, to our knowledge, this is the first attempt to use a per sample per label weighted loss for missing labels where the missing labels are unknown.…”

“…The set of missing labels is often not known. Hence, this problem of MLML is different from the problem of partial labels [11], where the position of the missing labels is known but its value is unknown, and noisy labels [31] where a set of both positive and negative labels are corrupted.…”

“…(1) Present a new weighted loss function that accounts for the confidence in the labels while training in a way that is easily usable to fine-tune pre-trained models. (2) A weighting schema per sample per label to estimate the presence of the missing labels, unlike previous work where the location of missing labels is assumed to be known [11,17] (3) We demonstrate the benefit of our approach on synthetic experimental setup by comparing the performance of the fine-tuned models with the weighted loss compared to nonweighted loss on two different datasets. The improvement is consistent across different ratios of missing labels 3 .…”

Classification of Hyperspectral Image Based on Double-Branch Dual-Attention Mechanism Network

“…Deep learning (DL) has shown powerful capabilities in automatically extracting nonlinear and hierarchical features. A great surge of computer vision tasks have benefited from DL and made significant breakthroughs, such as objection detection [18], natural language processing [19], and image classification [20]. As a typical classification tasks, HSI classification has been deeply influenced by DL and has obtained excellent improvements.…”

Extracting Raft Aquaculture Areas from Remote Sensing Images via an Improved U-Net with a PSE Structure

“…In the field of deep learning, the CNN (Convolutional Neural Network), R-CNN (Regions with Convolutional Neural Network features) and FCN (Fully Convolutional Network) models are rapidly developing and evolving respectively for image classification, target detection and semantic segmentation tasks. For examples, Thibaut et al proposed an end-to-end method to learn a multi-label classifier with partial labels [14]. They introduced a loss function that generalizes the standard binary cross-entropy loss by exploiting label proportion information and significantly improves performance on multi-label classification.…”