Stable Transductive Learning

Abstract

Abstract. We develop a new error bound for transductive learning algorithms. The slack term in the new bound is a function of a relaxed notion of transductive stability, which measures the sensitivity of the algorithm to most pairwise exchanges of training and test set points. Our bound is based on a novel concentration inequality for symmetric functions of permutations. We also present a simple sampling technique that can estimate, with high probability, the weak stability of transductive learning algorithms with respect to a given dataset. We demonstrate the usefulness of our estimation technique on a well known transductive learning algorithm.

Stability of transductive regression algorithms

“…2 we give a formal definition of the transductive regression setting and the notion of stability for transduction. Our bounds generalize the stability bounds given by Bousquet and Elisseeff (2002) for the inductive setting and extend to regression the stabilitybased transductive classification bounds of (El-Yaniv & Pechyony, 2006). Standard concentration bounds such as McDiarmid's bound (McDiarmid, 1989) cannot be readily applied to the transductive regression setting since the points are not drawn independently but uniformly without replacement from a finite set.…”

“…Standard concentration bounds such as McDiarmid's bound (McDiarmid, 1989) cannot be readily applied to the transductive regression setting since the points are not drawn independently but uniformly without replacement from a finite set. Instead, a generalization of McDiarmid's bound that holds for random variables sampled without replacement is used, as in (El-Yaniv & Pechyony, 2006). Sec.…”

Error bounds of multi-graph regularized semi-supervised classification

“…As illustrated in [5,1], the graph Laplacian regularization from the perspective of the graph smoothing functional is equivalent to a kernel regularization of a reproducing kernel Hilbert space (RKHS). However, as shown in [13], the algorithms considered in [1,5] have the deficiency that they always have unnecessary constrains for the labels of the vertices V. To remedy this deficiency, we introduce the following graph-based quantity…”

Generalization performance of graph-based semi-supervised classification

“…However, as pointed in [19], the algorithms considered in [4,14] have unnecessary constrains for the labels of the vertices. In order to remedy this deficiency, we introduce the following graph-based quantity…”

“…Firstly, we design the graph-based semi-supervised algorithm without any constrains on class proportion (see [19]). It has long been noticed that constraining the class proportions on unlabeled data can be important for semi-supervised learning.…”