Gentle Nearest Neighbors Boosting over Proper Scoring Rules

Nock, Richard; Ali, Wafa Bel Haj; D'Ambrosio, Roberto; Nielsen, Franck; Barlaud, Michel

doi:10.1109/tpami.2014.2307877

Cited by 6 publications

(4 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some exemplary (relatively) recent applications of generally non-separable (ordinary/classical) Bregman distances appear e.g. in Jiao et al [206], Varshney & Varshney [324], Hu et al [325], Nock et al [326], Raskutti & Mukherjee [327], Wang et al [328], He et al [329], Li et al [330], Harremoës [331], Xu et al [332], Halder [333], Zhang et al [334], Shao et al [335], Tembine [336], Brécheteau et al [337], Lin et al [338], Yuan et al [339], Azizan et al [340], Dytso et al [341], Gruzdeva & Ushakov [342], Song et al [343], Tan & Zhang [245], Yu et al [344], Capó et al [345], Chen et al [346], Fernández-Rodriguez [347], Hayashi [348], Li & Ralescu [349], Xiong et al [350], Liu et al [351].…”

Section: A Further Divergences and Friendsmentioning

confidence: 99%

A Precise Bare Simulation Approach to the Minimization of Some Distances. I. Foundations

Broniatowski¹,

Stummer

2023

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

The constrained minimization (respectively maximization) of directed distances and of related generalized entropies is a fundamental task in information theory as well as in the adjacent fields of statistics, machine learning, artificial intelligence, signal processing and pattern recognition. In our previous paper "A precise bare simulation approach to the minimization of some distances. I. Foundations", we obtained such kind of constrained optima by a new dimension-free precise bare (pure) simulation method, provided basically that (i) the underlying directed distance is of f −divergence type, and that (ii) this can be connected to a light-tailed probability distribution in a certain manner. In the present paper, we extend this approach such that constrained optimization problems of a very huge amount of directed distances and generalized entropies -and beyond -can be tackled by a newly developed dimension-free extended bare simulation method, for obtaining both optima as well as optimizers. Almost no assumptions (like convexity) on the set of constraints are needed, within our discrete setup of arbitrary dimension, and our method is precise (i.e., converges in the limit). For instance, we cover constrained optimizations of arbitrary f −divergences, Bregman distances, scaled Bregman distances and weighted ℓr−distances. The potential for wide-spread applicability is indicated, too; in particular, we deliver many recent references for uses of the involved distances/divergences in various different research fields (which may also serve as an interdisciplinary interface).

show abstract

Section: A Further Divergences and Friendsmentioning

confidence: 99%

A Precise Bare Simulation Approach to the Minimization of Some Distances. I. Foundations

Broniatowski¹,

Stummer

2023

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

show abstract

“…Although there are many machine learning methods for feature selection such as LASSO [ 14 , 15 ], Discriminant analysis [ 16 ], Proximal methods [ 17 , 18 ] and Boosting [ 19 , 20 ], here we compare our novel Primal-Dual method for Classification with Rejection (PD-CR) to the state of the art PLS-DA and Random Forests and SVM classification methods frequently used in metabolomics studies.…”

Section: Introductionmentioning

confidence: 99%

Primal-dual for classification with rejection (PD-CR): a novel method for classification and feature selection—an application in metabolomics studies

et al. 2021

Self Cite

View full text Add to dashboard Cite

Background Supervised classification methods have been used for many years for feature selection in metabolomics and other omics studies. We developed a novel primal-dual based classification method (PD-CR) that can perform classification with rejection and feature selection on high dimensional datasets. PD-CR projects data onto a low dimension space and performs classification by minimizing an appropriate quadratic cost. It simultaneously optimizes the selected features and the prediction accuracy with a new tailored, constrained primal-dual method. The primal-dual framework is general enough to encompass various robust losses and to allow for convergence analysis. Here, we compare PD-CR to three commonly used methods: partial least squares discriminant analysis (PLS-DA), random forests and support vector machines (SVM). We analyzed two metabolomics datasets: one urinary metabolomics dataset concerning lung cancer patients and healthy controls; and a metabolomics dataset obtained from frozen glial tumor samples with mutated isocitrate dehydrogenase (IDH) or wild-type IDH. Results PD-CR was more accurate than PLS-DA, Random Forests and SVM for classification using the 2 metabolomics datasets. It also selected biologically relevant metabolites. PD-CR has the advantage of providing a confidence score for each prediction, which can be used to perform classification with rejection. This substantially reduces the False Discovery Rate. Conclusion PD-CR is an accurate method for classification of metabolomics datasets which can outperform PLS-DA, Random Forests and SVM while selecting biologically relevant features. Furthermore the confidence score provided with PD-CR can be used to perform classification with rejection and reduce the false discovery rate.

show abstract

“…Although there are many machine learning methods for feature selection such as support vector machines (SVM) [11], LASSO [12,13], Discriminant analysis [14], Proximal methods [15,16], Boosting [17,18], we compare here our novel Primal-Dual method for Classification with Rejection (PD-CR) to the state of the art PLS-DA and Random Forests classification methods used in metabolomics studies, available with the popular Metaboanalyst 5.0 (www.metaboanalyst.ca) [19].…”

Section: Introductionmentioning

confidence: 99%

Primal-Dual for Classification with Rejection (PD-CR): A Novel Method for Classification and Feature Selection. An Application in Metabolomics Studies.

Chardin

Humbert

Bailleux

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Background: Supervised classification methods have been used for many years for feature selection in metabolomics and other omics studies. We developed a novel primal-dual based classification method (PD-CR) that can perform classification with rejection and feature selection on high dimensional datasets. PD-CR projects data onto a low dimension space and performs classification by minimizing an appropriate quadratic cost. It simultaneously optimizes the selected features and the prediction accuracy with a new tailored, constrained primal-dual method. The primal-dual framework is general enough to encompass various robust losses and to allow for convergence analysis. Here, we compared PD-CR to two commonly used methods : Partial Least Squares Discriminant Analysis (PLS-DA) and Random Forests. We analyzed two metabolomics datasets: one urinary metabolomics dataset concerning lung cancer patients and healthy controls; and a metabolomics dataset obtained from frozen glial tumor samples with mutated isocitrate dehydrogenase (IDH) or wild-type IDH. Results: PD-CR was more accurate than PLS-DA and Random Forests for classification using the 2 metabolomics datasets. It also selected biologically relevant metabolites. PD-CR has the advantage of providing a confidence score for each prediction, which can be used to perform classification with rejection. This substantially reduces the False Discovery Rate. Conclusion: The confidence score provided with PD-CR adds considerable value to the prediction as it includes a metric that is implicitly used by every physician when they make a medical decision: the probability to make the wrong choice. So far, one of the main obstacles to the use of machine learning in medicine resides in the fact that it is harder to trust the decision of a machine learning method than that of a physician when it comes to health issues. We believe that providing a confidence score associated to the decision would make these new tools more convincing if used in routine clinical practice.

show abstract

Gentle Nearest Neighbors Boosting over Proper Scoring Rules

Cited by 6 publications

References 32 publications

A Precise Bare Simulation Approach to the Minimization of Some Distances. I. Foundations

A Precise Bare Simulation Approach to the Minimization of Some Distances. I. Foundations

Primal-dual for classification with rejection (PD-CR): a novel method for classification and feature selection—an application in metabolomics studies

Primal-Dual for Classification with Rejection (PD-CR): A Novel Method for Classification and Feature Selection. An Application in Metabolomics Studies.

Contact Info

Product

Resources

About