Stability of transductive regression algorithms

Cortes, Corinna; Mohri, Mehryar; Pechyony, Dmitry; Rastogi, Ashish

doi:10.1145/1390156.1390179

Cited by 26 publications

(35 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Different from the existing notions of stability, see, e.g., [31,16,5,21,29,12,34] and the references therein, our proposed stability is defined on the kernel matrix. Therefore, we can estimate its value from empirical data, which makes this stability usable for kernel selection in practice.…”

Section: Definition 1 (Kernel Stability) a Kernel Function K Is Of βmentioning

confidence: 99%

Preventing Over-Fitting of Cross-Validation with Kernel Stability

Liu

Liao

2014

Machine Learning and Knowledge Discovery in Databases

View full text Add to dashboard Cite

Abstract. Kernel selection is critical to kernel methods. Cross-validation (CV) is a widely accepted kernel selection method. However, the CV based estimates generally exhibit a relatively high variance and are therefore prone to over-fitting. In order to prevent the high variance, we first propose a novel version of stability, called kernel stability. This stability quantifies the perturbation of the kernel matrix with respect to the changes in the training set. Then we establish the connection between the kernel stability and variance of CV. By restricting the derived upper bound of the variance, we present a kernel selection criterion, which can prevent the high variance of CV and hence guarantee good generalization performance. Furthermore, we derive a closed form for the estimate of the kernel stability, making the criterion based on the kernel stability computationally efficient. Theoretical analysis and experimental results demonstrate that our criterion is sound and effective.

show abstract

Section: Definition 1 (Kernel Stability) a Kernel Function K Is Of βmentioning

confidence: 99%

Preventing Over-Fitting of Cross-Validation with Kernel Stability

Liu

Liao

2014

Machine Learning and Knowledge Discovery in Databases

View full text Add to dashboard Cite

show abstract

“…However, classification in heterogeneous networks [1–4] and regression in homogeneous networks [5, 6] have been studied. Some homogeneous and heterogeneous graph-based classification methods do provide numerical ‘soft’ predictions before assigning the class labels [1, 2, 7].…”

Section: Introductionmentioning

confidence: 99%

Graph Regularized Meta-path Based Transductive Regression in Heterogeneous Information Network

Wan

Ouyang

Kaplan

2015

Proceedings of the 2015 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

A number of real-world networks are heterogeneous information networks, which are composed of different types of nodes and links. Numerical prediction in heterogeneous information networks is a challenging but significant area because network based information for unlabeled objects is usually limited to make precise estimations. In this paper, we consider a graph regularized meta-path based transductive regression model (Grempt), which combines the principal philosophies of typical graph-based transductive classification methods and transductive regression models designed for homogeneous networks. The computation of our method is time and space efficient and the precision of our model can be verified by numerical experiments.

show abstract

“…Harmonic solution and consistency method are the instances of a bigger class of the optimization problems called the unconstrained regularization [22]. In the transductive setting, the unconstrained regularization searches for soft (continuous) label assignment such that it maximizes fit to the labeled data and penalizes for not following the manifold structure:

{bold-italicℓ}^{★} = min_{ℓ \in ℝ^{n}} {(ℓ - y)}^{normalT} C (ℓ - y) + {bold-italicℓ}^{normalT} K ℓ,

where K is a symmetric regularization matrix and C is a symmetric matrix of empirical weights.…”

Section: Introductionmentioning

confidence: 99%

“…Consistency method has K equal to the normalized graph Laplacian K = I − D −1/2 WD −1/2 and c u = c l is set to a non-zero constant. The appealing property of (1) is that its solution can be computed in closed form as follows [22]:

{bold-italicℓ}^{★} = {(C^{- 1} K + I)}^{- 1} y

…”

Section: Introductionmentioning

confidence: 99%

Conditional Anomaly Detection with Soft Harmonic Functions

Valko

Kveton

Valizadegan

et al. 2011

2011 IEEE 11th International Conference on Data Mining

View full text Add to dashboard Cite

In this paper, we consider the problem of conditional anomaly detection that aims to identify data instances with an unusual response or a class label. We develop a new non-parametric approach for conditional anomaly detection based on the soft harmonic solution, with which we estimate the confidence of the label to detect anomalous mislabeling. We further regularize the solution to avoid the detection of isolated examples and examples on the boundary of the distribution support. We demonstrate the efficacy of the proposed method on several synthetic and UCI ML datasets in detecting unusual labels when compared to several baseline approaches. We also evaluate the performance of our method on a real-world electronic health record dataset where we seek to identify unusual patient-management decisions.

show abstract

Stability of transductive regression algorithms

Cited by 26 publications

References 9 publications

Preventing Over-Fitting of Cross-Validation with Kernel Stability

Preventing Over-Fitting of Cross-Validation with Kernel Stability

Graph Regularized Meta-path Based Transductive Regression in Heterogeneous Information Network

Conditional Anomaly Detection with Soft Harmonic Functions

Contact Info

Product

Resources

About