When Does Cotraining Work in Real Data?

Abstract

Abstract-Co-training, a paradigm of semi-supervised learning, is promised to alleviate effectively the shortage of labeled examples in supervised learning. The standard two-view co-training requires the dataset to be described by two views of features, and previous studies have shown that co-training works well if the two views satisfy the sufficiency and independence assumptions. In practice, however, these two assumptions are often not known or ensured (even when the two views are given). More commonly, most supervised datasets are described by one set of attributes (one view). Thus, they need be split into two views in order to apply the standard twoview co-training. In this paper, we first propose a novel approach to empirically verify the two assumptions of co-training given two views. Then, we design several methods to split single view datasets into two views, in order to make co-training work reliably well. Our empirical results show that, given a whole or a large labeled training set, our view verification and splitting methods are quite effective. Unfortunately, co-training is called for precisely when the labeled training set is small. However, given small labeled training sets, we show that the two co-training assumptions are difficult to verify, and view splitting is unreliable. Our conclusions for co-training's effectiveness are mixed. If two views are given, and known to satisfy the two assumptions, co-training works well. Otherwise, based on small labeled training sets, verifying the assumptions or splitting single view into two views are unreliable, thus it is uncertain whether the standard co-training would work or not.

SEG-SSC: A Framework Based on Synthetic Examples Generation for Self-Labeled Semi-Supervised Classification

“…Nevertheless, this method has become an example for recent models thanks to the idea of using the agreement (or disagreement) of multiple classifiers and the mutual teaching approach. A good study of when co-training works can be found in [32].…”

Semi-Supervised Learning on Single-View Datasets by Integration of Multiple Co-trained Classifiers

“…Some promising results have been achieved in this field [3][4][5][6][7], but this proved to be a difficult task, as the relation between the characteristics of the views and the performance of cotraining has not been sufficiently understood. Moreover, research [4] indicates that given a small training dataset as in real-world situations where co-training is called for, the sufficiency and independence assumptions cannot be reliably verified, making the split methods unreliable and application of co-training uncertain.…”

“…In addition, we performed experiments on 14 binary and 8 multi-class UCI datasets also previously used for evaluating co-training [4,8,13]. The benchmark datasets of various properties were selected to give us a better insight of how effective our method is on datasets of various dimensionality, size and redundancy.…”

“…All groups are sorted by the parameter Gap. Notation -Dataset: the number in parentheses following the dataset names denotes the number of classes; Dim: the number of features describing the dataset; |L|: the size of the initial training set L; L acc : accuracy achieved by a supervised Naive Bayes classifier trained on the initial set L; |All|: the size of the entire training set All (i.e., the sum of numbers of labeled and unlabeled examples); All acc : accuracy achieved by supervised Naive Bayes classifier trained on the entire training set All (i.e., labeled examples and unlabeled examples with correct label); Gap: performance gap (also called the optimal gain in [4]) computed as All acc -L acc .…”

Unlabeled Data and Multiple Views

“…It is evident, however, that a random split would not work in most cases. Du et al [8] tried several heuristics for view split and found that all heuristics failed with insufficient labeled data. The necessary condition of co-training given in [24] suggested that among all potential view splits, the one which enables the most unlabeled instances connect with labeled examples in the combinative graph is preferred; this was empirically verified in [24] and might give inspiration to develop sound practical view split approaches.…”