Weighted quality estimates in machine learning

Abstract

We suggested a novel approach to calculate prediction model quality based on assigning to each data point inverse density weights derived from the postulated distance metric. We demonstrated that our new weighted measures estimate the model generalization better and are consistent with the machine learning theory. The Vapnik-Chervonenkis theorem was reformulated and applied to derive the space-uniform error estimates. Two examples were used to illustrate the advantages of the inverse density weighting. First, we demonstrated on a set with a built-in bias that the unweighted cross-validation procedure leads to an overly optimistic quality estimate, while the density-weighted quality estimates are more realistic. Second, an analytical equation for weighted quality estimates was used to derive an SVM model for signal peptide prediction using a full set of known signal peptides, instead of the usual filtered subset.

An artificial intelligence-based risk prediction model of myocardial infarction

“…Model 5 was constructed by combining the w-easy ensemble with RFE. In machine learning, weighted methods can be used to eliminate training biases [ 20 ]. W in the W-easy ensemble means weight.…”

Prediction of One‐Dimensional Structural Properties Of Proteins by Integrated Neural Networks

“…density -weighted quality estimates for better generalization [94] , bidirectional segmented -memory recurrent neural network for nonlocal interactions [95] , large -scale recursive neural network [96] , and guided learning in Real -SPINE [4] .…”

“…This includes variants of Support Vector Machine models [52 -59] , variants of neural networks [60 -69] , cascaded nonlinear components analysis [70] , the consensus or ensemble of multiple predictors [64,65,71 -81] , Bayesian or hidden semi -Markov network [82 -85] , multiple linear regression [86,87] , k -nearest neighborhood [88] , dynamic programming algorithm [89] , hybrid genetic -neural system [90] , sequence pre -clustering [91,92] , and conditional random fi elds for combined prediction [77] . Some notable new proposals are Bayesian neural network rather than commonly used backpropagation neural networks [93] , density -weighted quality estimates for better generalization [94] , bidirectional segmented -memory recurrent neural network for nonlocal interactions [95] , large -scale recursive neural network [96] , and guided learning in Real -SPINE [4] .…”

Exploring classification strategies with the CoEPrA 2006 contest

“…The winner of the second CoEPrA classification task is the group of Levon Budagyan obtaining an MCC of 0.7108. They used a support vector machine (SVM) (Boser et al, 1992;Vapnik, 1995) classifier together with gapped pair counts as descriptors (Budagyan and Abagyan, 2006). Again with a combination of sparse and physico-chemical features the MSE classifier yields results of the same quality.…”