Transductive Label Augmentation for Improved Deep Network Learning

Abstract

A major impediment to the application of deep learning to real-world problems is the scarcity of labeled data. Small training sets are in fact of no use to deep networks as, due to the large number of trainable parameters, they will very likely be subject to overfitting phenomena. On the other hand, the increment of the training set size through further manual or semi-automatic labellings can be costly, if not possible at times. Thus, the standard techniques to address this issue are transfer learning and data augmentation, which consists of applying some sort of "transformation" to existing labeled instances to let the training set grow in size. Although this approach works well in applications such as image classification, where it is relatively simple to design suitable transformation operators, it is not obvious how to apply it in more structured scenarios. Motivated by the observation that in virtually all application domains it is easy to obtain unlabeled data, in this paper we take a different perspective and propose a label augmentation approach. We start from a small, curated labeled dataset and let the labels propagate through a larger set of unlabeled data using graph transduction techniques. This allows us to naturally use (second-order) similarity information which resides in the data, a source of information which is typically neglected by standard augmentation techniques. In particular, we show that by using known game theoretic transductive processes we can create larger and accurate enough labeled datasets which use results in better trained neural networks. Preliminary experiments are reported which demonstrate a consistent improvement over standard image classification datasets.

Label Propagation for Deep Semi-Supervised Learning

“…θ ← OPTIMIZE(Lw(X, Y L ,Ŷ U ; θ)) mini-batch optimization 17: end for 18: end procedure Our main idea therefore is that instead of just encouraging nearby examples to get the same predictions, we encourage all examples to get predictions same as the ones we would get by transductive learning according to the quadratic cost (8) and its solution Z (6). Computing Z is efficient because it is performed outside our main optimization process, i.e.…”

“…Learning by association [17] can been seen as two steps of propagation on a constrained bi-partite graph between labeled and unlabeled examples. Graph transduction game (GTG) [9], a form of label propagation, has been used for pseudolabels [8] as in our work, but in this case the network is pre-trained, the graph remains fixed and there is no weighting mechanism. We compare to this approach in Section 5.…”

“…To assess the benefit of diffusion, we finally evaluate a variant of our approach where the pseudo-labels are not provided by diffusion but derived from the network with (1) or from GTG propagation [8] instead. Training is performed with (12), as with our method.…”

Two sides of the same coin: Improved ancient coin classification using Graph Transduction Games

“…The Graph Transduction Game (GTG) [6], is a semisupervised learning method which has recently found a renewed interest and successfully applied in different contexts, e.g. bioinformatics [19] and label augmentation problem [5]. The GTG casts the problem in terms of a non-cooperative multiplayer game, in which the objects (or images of a dataset) are the players while the possible strategies are the class labels.…”

Unsupervised Domain Adaptation using Graph Transduction Games

“…This lack of generalization can be easily overcome through the training of a supervised classifier with the newly labeled dataset (cf. [3]). In GT the data is modeled as a graph G = (V, E, w) whose vertices are the observations in a dataset and edges represent similarities among them.…”