Statistical analysis of kernel-based least-squares density-ratio estimation

Kanamori, Takafumi; Suzuki, Taiji; Sugiyama, Masashi

doi:10.1007/s10994-011-5266-3

Cited by 88 publications

(111 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Among these direct density ratio estimators, an unconstrained version of LSIF (uLSIF) with a kernel density- ratio model was demonstrated to be highly useful in terms of both accuracy [8] and computational efficiency [9]. For that reason, uLSIF-based machine learning algorithms have been successfully used in solving various machine learning tasks [10]- [12].…”

Section: Introductionmentioning

confidence: 99%

Direct Density Ratio Estimation with Convolutional Neural Networks with Application in Outlier Detection

Nam

Sugiyama

2015

IEICE Trans. Inf. & Syst.

Self Cite

View full text Add to dashboard Cite

SUMMARYRecently, the ratio of probability density functions was demonstrated to be useful in solving various machine learning tasks such as outlier detection, non-stationarity adaptation, feature selection, and clustering. The key idea of this density ratio approach is that the ratio is directly estimated so that difficult density estimation is avoided. So far, parametric and non-parametric direct density ratio estimators with various loss functions have been developed, and the kernel least-squares method was demonstrated to be highly useful both in terms of accuracy and computational efficiency. On the other hand, recent study in pattern recognition exhibited that deep architectures such as a convolutional neural network can significantly outperform kernel methods. In this paper, we propose to use the convolutional neural network in density ratio estimation, and experimentally show that the proposed method tends to outperform the kernelbased method in outlying image detection. key words: density ratio estimation, convolutional neural network, outlier detection

show abstract

Section: Introductionmentioning

confidence: 99%

Direct Density Ratio Estimation with Convolutional Neural Networks with Application in Outlier Detection

Nam

Sugiyama

2015

IEICE Trans. Inf. & Syst.

Self Cite

View full text Add to dashboard Cite

show abstract

“…As such, this approach can be computationally cheaper than kernel mean matching. It can also be extended to be kernel-based [106]. There are also direct weight estimators that do not employ optimization.…”

Section: Data Importance-weightingmentioning

confidence: 99%

A Review of Domain Adaptation without Target Labels

Kouw

Loog

2021

IEEE Trans. Pattern Anal. Mach. Intell.

380

213

View full text Add to dashboard Cite

Domain adaptation has become a prominent problem setting in machine learning and related fields. This review asks the question: how can a classifier learn from a source domain and generalize to a target domain? We present a categorization of approaches, divided into, what we refer to as, sample-based, feature-based and inference-based methods. Sample-based methods focus on weighting individual observations during training based on their importance to the target domain. Feature-based methods revolve around on mapping, projecting and representing features such that a source classifier performs well on the target domain and inference-based methods incorporate adaptation into the parameter estimation procedure, for instance through constraints on the optimization procedure. Additionally, we review a number of conditions that allow for formulating bounds on the cross-domain generalization error. Our categorization highlights recurring ideas and raises questions important to further research.

show abstract

“…The goal of density ratio estimation is to find an accurate estimate w of the density ratio function w using patterns X nu and X de [34]. There are many existing methods that can be applied for estimating the density ratio function [14,21,23,33,37]. To the best of our knowledge, there is no experiment that applies density ratio estimation for PU classification.…”

Section: Density Ratio Estimationmentioning

confidence: 99%

Positive-Unlabeled Classification under Class Prior Shift and Asymmetric Error

Charoenphakdee¹,

Sugiyama²

2019

Proceedings of the 2019 SIAM International Conference on Data Mining

Self Cite

View full text Add to dashboard Cite

Bottlenecks of binary classification from positive and unlabeled data (PU classification) are the requirements that given unlabeled patterns are drawn from the test marginal distribution, and the penalty of the false positive error is identical to the false negative error. However, such requirements are often not fulfilled in practice. In this paper, we generalize PU classification to the class prior shift and asymmetric error scenarios. Based on the analysis of the Bayes optimal classifier, we show that given a test class prior, PU classification under class prior shift is equivalent to PU classification with asymmetric error. Then, we propose two different frameworks to handle these problems, namely, a risk minimization framework and density ratio estimation framework. Finally, we demonstrate the effectiveness of the proposed frameworks and compare both frameworks through experiments using benchmark datasets.

show abstract

Statistical analysis of kernel-based least-squares density-ratio estimation

Cited by 88 publications

References 33 publications

Direct Density Ratio Estimation with Convolutional Neural Networks with Application in Outlier Detection

Direct Density Ratio Estimation with Convolutional Neural Networks with Application in Outlier Detection

A Review of Domain Adaptation without Target Labels

Positive-Unlabeled Classification under Class Prior Shift and Asymmetric Error

Contact Info

Product

Resources

About