Bayesian and pairwise local similarity discriminant analysis

Abstract

We investigate three extensions to the generative similarity-based classifier called local similarity discriminant analysis (local SDA): a Bayesian approach to estimating the pmfs based on the assumption that similarities are multinomially distributed and on the Dirichlet prior distribution; a pairwisesimilarity formulation of local SDA that accounts for all local pairwise similarities to estimate the pmfs; a combined Bayesian pairwise-similarity approach. We discuss how the proposed extensions afford more modeling flexibility than standard local SDA and less cumbersome model training than previouslypublished local SDA regularization strategies. Experiments with five benchmark similarity-based classification datasets show that the increased modeling flexibility and lighter computational burden of the proposed extensions are coupled with the good classification performance of the local SDA classification paradigm.Index Terms-similarity-based classification; discriminant analysis; Bayesian; prototype; Dirichlet distribution; I. SIMILARITY-BASED CLASSIFICATIONSimilarity-based classifiers learn from a set of pairwise training similarities, training class labels, and from the similarities between a test sample and the training samples [1]. Similarity-based classifiers are independent of a chosen similarity measure, which is usually problem-dependent and can subsume complex relationships between complex, heterogeneous samples. In this paper, we focus on the problem of designing generative classifiers for similarity-based learning. Here, the goal is to create class-conditional probabilistic models of the given similarities. Generative similarity-based classifiers differ from the standard metric-based generative classifiers, such as quadratic discriminant analysis and Gaussian mixture models, because the modeled quantity is the pairwise similarity between the samples rather than the numerical feature vectors that describe the samples. Producing class probabilities is important in many practical systems where there may be skewed class priors or asymmetric misclassification costs, or where probabilities are required as an input to the next component in the system or to fuse with probabilistic information about the class label derived from other sources.Recently, an effective generative classifier for similaritybased learning called similarity discriminant analysis (SDA) was proposed [2], followed by a local version (local SDA) [3], and a regularized local version [4]. We review the standard local SDA classifier in Section II, and discuss its limitations.In Section III, we introduce the first contribution of this paper: a Bayesian framework for estimating the local SDA class-conditional pmfs based on a the assumption that the similarities are multinomially distributed and on a Dirichletdistributed prior. In Section IV we introduce the second contribution of this paper: a pairwise local SDA classifier which endows local SDA with increased flexibility and robustness. We also discuss a combined pairwise, Bayesian approach to m...