Taming the Curse of Dimensionality in Kernels and Novelty Detection

Evangelista, Paul; Embrechts, Mark J.; Szymanski, Boleslaw K.

doi:10.1007/3-540-31662-0_33

Cited by 42 publications

(39 citation statements)

References 20 publications

(9 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The deterioration of the performance of Kernel_MEPSO is in accordance with (Evangelista et al, 2006), which indicates that higher dimensionality confounds the process of kernel based learning especially in presence of unbalanced classes. The curse of dimensionality may be overcome in Kernel based clustering by utilizing the subspace models.…”

Section: Scalability Of the Kernel_mepso Algorithmmentioning

confidence: 52%

Automatic kernel clustering with a Multi-Elitist Particle Swarm Optimization Algorithm

Das

Abraham

Konar

2008

Pattern Recognition Letters

174

View full text Add to dashboard Cite

This article introduces a scheme for clustering complex and linearly non-separable datasets, without any prior knowledge of the number of naturally occurring groups in the data. The proposed method is based on a modified version of classical Particle Swarm Optimization (PSO) algorithm, known as the Multi-Elitist PSO (MEPSO) model. It also employs a kernel-induced similarity measure instead of the conventional sum-of-squares distance. Use of the kernel function makes it possible to cluster data that is linearly non-separable in the original input space into homogeneous groups in a transformed high-dimensional feature space. A new particle representation scheme has been adopted for selecting the optimal number of clusters from several possible choices. The performance of the proposed method has been extensively compared with a few state of the art clustering techniques over a test suit of several artificial and real life datasets. Based on the computer simulations, some empirical guidelines have been provided for selecting the suitable parameters of the PSO algorithm.

show abstract

Section: Scalability Of the Kernel_mepso Algorithmmentioning

confidence: 52%

Automatic kernel clustering with a Multi-Elitist Particle Swarm Optimization Algorithm

Das

Abraham

Konar

2008

Pattern Recognition Letters

174

View full text Add to dashboard Cite

show abstract

“…In complex problems, the number of parameters that need to be set within the cognitive model is large and cannot be readily identified even with large training data sets. Because of the range of possible actions, these problems suffer from the "curse of dimensionality" where even sophisticated machine learning approaches and parameter tuning alone will not suffice (Evangelista et al, 2006;Bengio et al, 2005). In these problems, the novel hybrid approach we present-using machine learning Combining Psychological Models with Machine Learning 17 algorithms as the base of the solution, but adding features from the cognitive models -creates significant improvements over both base approaches, often by large amounts.…”

Section: Discussionmentioning

confidence: 99%

“…These domains suffer from a phenomenon often known as the "curse of dimensionality" (Evangelista, Embrechts, & Szymanski, 2006;Bengio, Delalleau, & Roux, 2005). The curse of dimensionality is, unfortunately, particularly evident in many real-world situations, as people can potentially act within a very large set of actions (a high dimension of possible actions), and they often do not consistently choose the same actions.…”

Section: Introductionmentioning

confidence: 99%

Combining psychological models with machine learning to better predict people’s decisions

et al. 2012

View full text Add to dashboard Cite

Creating agents that proficiently interact with people is critical for many applications. Towards creating these agents, models are needed that effectively predict people's decisions in a variety of problems. To date, two approaches have been suggested to generally describe people's decision behavior. One approach creates a-priori predictions about people's behavior, either based on theoretical rational behavior or based on psychological models, including bounded rationality. A second type of approach focuses on creating models based exclusively on observations of people's behavior. At the forefront of these types of methods are various machine learning algorithms. This paper explores how these two approaches can be compared and combined in different types of domains. In relatively simple domains, both psychological models and machine learning yield clear prediction models with nearly identical results. In more complex domains, the exact action predicted by psychological models is not even clear, and machine learning models are even less accurate. Nonetheless, we present a novel approach of creating hybrid methods that incorporate features from psychological models in conjunction with machine learning in order to create significantly improved models for predicting people's decisions. To demonstrate these claims, we present an overview of previous and new results, taken from representative domains ranging from a relatively simple optimization problem and complex domains such as negotiation and coordination without communication.

show abstract

“…In addition, we use random forest (RF) both for feature selection and to benchmark the performance of variants of kNN. RF is a commonly used robust classifier which can handle large amount of features without being affected by the curse of dimensionality due to inherent internal randomised distribution of data among decision trees [86].…”

Section: B the Learnersmentioning

confidence: 99%

“…The nearest neighbour learners are particularly sensitive to the curse of dimensionality [39] especially with small k values. High dimensionality is not a problem with RF due to distribution of randomised subsets of features and samples among decision trees [86]. Therefore, it is reasonable to ask that if we do feature selection before classification, is it likely that we will get better results?…”

Section: Tile-rfmentioning

confidence: 99%

Multiple Instance Learning for Breast Cancer Magnetic Resonance Imaging

Maken¹

View full text Add to dashboard Cite

In this thesis we evaluate the efficacy of multiple instance learning (MIL) as a 'pure' machine learning approach for the diagnosis of breast cancer in magnetic resonance images (MRI). The traditional approach for the diagnosis of breast cancer is based on region-of-interest (ROI) based single instance learning (SIL). In the ROI-based SIL, the classification of benign and malignant lesions depends on the features, which are extracted from segmented ROIs. But, an accurate segmentation of a ROI is a challenging task due to poor signal-to-noise-ratio and faint edges due to partial volume effects. Therefore, variations in the segmentation of ROIs can affect the diagnostic outcome. MIL is a relatively new method in the supervised learning, where each sample is represented as a bag of instances. An image in the context of MIL can be considered as a bag of pixels, tiles, or ROIs, which correspond to instances. Here we apply tile-based MIL, where the diagnosis of breast cancer is based on tile-based features which do not require the segmentation of ROIs. Therefore, tile-based MIL has the potential to provide the classification of breast cancer without segmenting the ROIs. This is the motivation for the main objective of this thesis: to estimate the efficacy of tile-based MIL in the detection and the diagnosis of breast cancer MRI.We initially evaluate the potential of MIL for the detection and the diagnosis of breast cancer in anatomical T2-weighted MRI. In particular, we compare the performance of a MIL-based learner, i.e., citation-kNN (CkNN) against conventional kNN and a Random Forest classifier. We utilise both (generic) tile-based spatial features and (domain specific) ROI-based features. We perform experiments on two datasets consisting of 77 mass-like lesions and 129 both mass-like and non-mass-like lesions. The performance of CkNN as both a diagnostic and screening tool is evaluated using the area under the receiver operating characteristic curve (AUC), estimated over 10-fold cross validation. Results demonstrate that the tile-based CkNN has equivalent performance to the ROI-based classification. However, tile-based MIL has an advantage that it does not require the domain specific ROI-based features typically used in breast MRI. This not only has the potential to make the tile-based classification robust to inaccuracies in the delineation of suspicious lesions, but also makes it suitable for the detection of suspicious lesions prior to segmentation.Next, we investigate the performance of CkNN for the diagnosis of breast cancer using dynamic MRI. Specifically we use generic tile-based spatio-temporal features derived from T2-weighted MRI and T1-weighted dynamic contrast enhanced MRI. We utilise a discrete cosine transform and contrast enhancement models as feature extraction techniques. We compare the III performance of CkNN and kNN against a traditional approach based on bespoke ROI-based features using the 77 mass-like lesions. Empirical results show the equivalent classification performance of both ti...

show abstract

Taming the Curse of Dimensionality in Kernels and Novelty Detection

Cited by 42 publications

References 20 publications

Automatic kernel clustering with a Multi-Elitist Particle Swarm Optimization Algorithm

Automatic kernel clustering with a Multi-Elitist Particle Swarm Optimization Algorithm

Combining psychological models with machine learning to better predict people’s decisions

Multiple Instance Learning for Breast Cancer Magnetic Resonance Imaging

Contact Info

Product

Resources

About