Perceptual Feature Identification for Active Sonar Echoes

2010 2nd International Workshop on Cognitive Information Processing

2010

We investigate three extensions to the generative similarity-based classifier called local similarity discriminant analysis (local SDA): a Bayesian approach to estimating the pmfs based on the assumption that similarities are multinomially distributed and on the Dirichlet prior distribution; a pairwisesimilarity formulation of local SDA that accounts for all local pairwise similarities to estimate the pmfs; a combined Bayesian pairwise-similarity approach. We discuss how the proposed extensions afford more modeling flexibility than standard local SDA and less cumbersome model training than previouslypublished local SDA regularization strategies. Experiments with five benchmark similarity-based classification datasets show that the increased modeling flexibility and lighter computational burden of the proposed extensions are coupled with the good classification performance of the local SDA classification paradigm.Index Terms-similarity-based classification; discriminant analysis; Bayesian; prototype; Dirichlet distribution; I. SIMILARITY-BASED CLASSIFICATIONSimilarity-based classifiers learn from a set of pairwise training similarities, training class labels, and from the similarities between a test sample and the training samples [1]. Similarity-based classifiers are independent of a chosen similarity measure, which is usually problem-dependent and can subsume complex relationships between complex, heterogeneous samples. In this paper, we focus on the problem of designing generative classifiers for similarity-based learning. Here, the goal is to create class-conditional probabilistic models of the given similarities. Generative similarity-based classifiers differ from the standard metric-based generative classifiers, such as quadratic discriminant analysis and Gaussian mixture models, because the modeled quantity is the pairwise similarity between the samples rather than the numerical feature vectors that describe the samples. Producing class probabilities is important in many practical systems where there may be skewed class priors or asymmetric misclassification costs, or where probabilities are required as an input to the next component in the system or to fuse with probabilistic information about the class label derived from other sources.Recently, an effective generative classifier for similaritybased learning called similarity discriminant analysis (SDA) was proposed [2], followed by a local version (local SDA) [3], and a regularized local version [4]. We review the standard local SDA classifier in Section II, and discuss its limitations.In Section III, we introduce the first contribution of this paper: a Bayesian framework for estimating the local SDA class-conditional pmfs based on a the assumption that the similarities are multinomially distributed and on a Dirichletdistributed prior. In Section IV we introduce the second contribution of this paper: a pairwise local SDA classifier which endows local SDA with increased flexibility and robustness. We also discuss a combined pairwise, Bayesian approach to m...

Section: A Data and Setupmentioning

confidence: 99%

Bayesian and pairwise local similarity discriminant analysis

Sadowski

Cazzanti

2010 2nd International Workshop on Cognitive Information Processing

2010

“…The Aural Sonar data set was developed to investigate the human ability to distinguish different types of sonar signals by ear (Philips et al, 2006), and consists of 100 samples. Each pairwise similarity is the sum of the similarity scores of two human subjects for that pair.…”

Section: Data Setsmentioning

confidence: 99%

Learning kernels from indefinite similarities

Chen

Proceedings of the 26th Annual International Conference on Machine Learning

Recht

2009

Similarity measures in many real applications generate indefinite similarity matrices. In this paper, we consider the problem of classification based on such indefinite similarities. These indefinite kernels can be problematic for standard kernel-based algorithms as the optimization problems become nonconvex and the underlying theory is invalidated. In order to adapt kernel methods for similarity-based learning, we introduce a method that aims to simultaneously find a reproducing kernel Hilbert space based on the given similarities and train a classifier with good generalization in that space. The method is formulated as a convex optimization problem. We propose a simplified version that can reduce overfitting and whose associated convex conic program can be solved efficiently. We compare the proposed simplified version with six other methods on a collection of real data sets.

“…Listeners perceptually evaluated the similarity between two sonar signals on a scale from 1 to 5. The pairwise similarities are the sum of the evaluations from two listeners, resulting in a perceptual similarity from 2 to 10 [8]. This dataset is interesting because perceptual similarities are often non-metric.…”

Section: A Similarity-based Problemsmentioning

confidence: 99%

Regularizing the Local Similarity Discriminant Analysis Classifier

Cazzanti

2009 International Conference on Machine Learning and Applications

2009

We investigate parameter-based and distributionbased approaches to regularizing the generative, similarity-based classifier called local similarity discriminant analysis classifier (local SDA). We argue that regularizing distributions rather than parameters can both increase the model flexibility and decrease estimation variance while retaining the conceptual underpinnings of the local SDA classifier. Experiments with four benchmark similarity-based classification datasets show that the proposed regularization significantly improves classification performance compared to the local SDA classifier, and the distributionbased approach improves performance more consistently than the parameter-based approaches. Also, regularized local SDA can perform significantly better than similarity-based SVM classifiers, particularly on sparse and highly nonmetric similarities.Keywords-local similarity discriminant analysis; regularized local similarity discriminant analysis; I. SIMILARITY-BASED CLASSIFICATIONSimilarity-based classifiers learn from a set of pairwise training similarities, training class labels, and from the similarities between a test sample and the training samples [1]. Similarity-based classifiers are independent of a chosen similarity measure, which is usually problem-dependent and can subsume complex relationships between complex, heterogeneous samples. In this paper, we focus on the problem of designing generative classifiers for similarity-based learning. Here, the goal is to create class-conditional probabilistic models of the given similarities. Generative similarity-based classifiers differ from the standard metric-based generative classifiers, such as quadratic discriminant analysis and Gaussian mixture models, because the modeled quantity is the pairwise similarities between the samples rather than the numerical feature vectors that describe the samples. Producing class probabilities is important in many practical systems where there may be skewed class priors or asymmetric misclassification costs, or where probabilities are required as an input to the next component in the system or to fuse with probabilistic information about the class label derived from other sources.Recently, an effective generative classifier for similaritybased learning called similarity discriminant analysis (SDA) Work supported by the U.S. Office of Naval Research and a local version (local SDA) were proposed [2], [3]. We review local SDA in Section 3, and discuss how this classifier can fail. In Section 4, we follow our analysis with a discussion of several regularization strategies for local SDA and with the main contribution of this paper: that appropriate regularization can both make the SDA model more flexible and lower the estimation variance. Experiments in Section 5 show that the proposed regularized local SDA improves on local SDA and can outperform other state-of-the-art similaritybased classifiers.Previous research on generative classifiers for similaritybased learning treated the n-vector of similarities between any ...