Linear Maximum Margin Classifier for Learning from Uncertain Data

Abstract

In this paper, we propose a maximum margin classifier that deals with uncertainty in data input. More specifically, we reformulate the SVM framework such that each training example can be modeled by a multi-dimensional Gaussian distribution described by its mean vector and its covariance matrix-the latter modeling the uncertainty. We address the classification problem and define a cost function that is the expected value of the classical SVM cost when data samples are drawn from the multi-dimensional Gaussian distributions that form the set of the training examples. Our formulation approximates the classical SVM formulation when the training examples are isotropic Gaussians with variance tending to zero. We arrive at a convex optimization problem, which we solve efficiently in the primal form using a stochastic gradient descent approach. The resulting classifier, which we name SVM with Gaussian Sample Uncertainty (SVM-GSU), is tested on synthetic data and five publicly available and popular datasets; namely, the MNIST, WDBC, DEAP, TV News Channel Commercial Detection, and TRECVID MED datasets. Experimental results verify the effectiveness of the proposed method.

A Classification Algorithm Based on Complex Number Feature

“…The Local Binary Pattern (LBP) feature and Speeded-Up Robust Feature (SURF) feature are extracted from object images. Then the proposed method is compared with three other sample uncertainty modeling methods, including uniform distribution [50], same Gaussian distribution [34], and features without considering the sample uncertainty with the same classifier SVM. The SVM hyper-parameters: penalty coefficient C=0.5, kernel function is radial basis function (RBF), the RBF coefficient gamma is 0.001.…”

“…The Wisconsin Diagnostic Breast Cancer (WDBC) dataset contains 569 examples, which can be divided into two classes: malignant (212 instances) and benign (357 instances The proposed method is compared with baseline SVM, power SVM [37], SVM-GSU [34], GEP-SVM [10] and FLST-SVM [9]. Because WDBC data does not provide a division in training and testing subsets, the dataset is divided into training subset (90%) and testing subset (10%).…”

“…To overcome these issues, this study utilizes two complex feature processing methods. The first method is the sample uncertainty modeling method of complex feature, which improves the classification effect of SVM [34]. Studies have illustrated the importance of sample uncertainty.…”

Interpretability of Multivariate Brain Maps in Linear Brain Decoding: Definition, and Heuristic Quantification in Multivariate Analysis of MEG Time-Locked Effects

“…The authors concluded that the interpretability of brain decoding cannot be improved using regularization. The problem is primarily caused by the decoding process per se , where it minimizes the classification error only considering the uncertainty in the output space (Zhang, 2005; Aggarwal and Yu, 2009; Tzelepis et al, 2015) and not the uncertainty in the input space (or noise). Our experimental results on the toy data (see Section 3.1) shows that if the right criterion is used for selecting the best values for hyper-parameters, appropriate choice of the regularization strategy can still play a significant role in improving the interpretability of results.…”

Interpretability of Multivariate Brain Maps in Brain Decoding: Definition and Quantification

“…Haufe et al [39] demonstrated that the weight in linear discriminative models are unable to accurately assess the relationship between independent variables, primarily because of the contribution of noise in the decoding process. The problem is primarily caused by the decoding process that minimizes the classification error only considering the uncertainty in the output space [80, 98, 99] and not the uncertainty in the input space (or noise). The authors concluded that the interpretability of brain decoding cannot be improved using regularization.…”