Multiresolution estimates of classification complexity

Singh, Sameer

doi:10.1109/tpami.2003.1251146

Cited by 64 publications

(30 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous attempts at analyzing the quality of the training set, or estimating the classification complexity of a given data set, are rather thoroughly summarized in [9]. Singh [9] suggests a multi-resolution analysis of the data by accumulating 'Purity' and 'Neighborhood Separability' of different partitionings of the data, and measures correlation of the measures, to the training and testing performance of some well known classifiers.…”

Section: Quantifying the Quality Of A Training Setmentioning

confidence: 99%

Section: Quantifying the Quality Of A Training Setmentioning

confidence: 99%

“…Singh [9] suggests a multi-resolution analysis of the data by accumulating 'Purity' and 'Neighborhood Separability' of different partitionings of the data, and measures correlation of the measures, to the training and testing performance of some well known classifiers. A detailed review of such methods is out of scope of this work and the reader is referred to [9] for that purpose. However, we highlight the following differences between our work and previous works such as [9]: 1) We use pairwise feature-selecting-similarity measures -described and motivated in section 2.1 -to describe the training data via some measures -described and motivated in section 2.2.…”

Section: Quantifying the Quality Of A Training Setmentioning

confidence: 99%

“…A detailed review of such methods is out of scope of this work and the reader is referred to [9] for that purpose. However, we highlight the following differences between our work and previous works such as [9]: 1) We use pairwise feature-selecting-similarity measures -described and motivated in section 2.1 -to describe the training data via some measures -described and motivated in section 2.2. Such an approach is acknowledged by [9] as 'a more sophisticated approach which requires further studies'.…”

Section: Quantifying the Quality Of A Training Setmentioning

confidence: 99%

“…However, we highlight the following differences between our work and previous works such as [9]: 1) We use pairwise feature-selecting-similarity measures -described and motivated in section 2.1 -to describe the training data via some measures -described and motivated in section 2.2. Such an approach is acknowledged by [9] as 'a more sophisticated approach which requires further studies'. 2) We explicitly model the interplay between features, similarity measures, classifiers, data, and test performance by linking the data-describing-measures to the test performance in section 2.3.…”

Section: Quantifying the Quality Of A Training Setmentioning

confidence: 99%

See 4 more Smart Citations

Properties of Datasets Predict the Performance of Classifiers

Aghazadeh

Carlsson

2013

Procedings of the British Machine Vision Conference 2013

View full text Add to dashboard Cite

It has been shown that the performance of classifiers depends not only on the number of training samples, but also on the quality of the training set [10,12]. The purpose of this paper is to 1) provide quantitative measures that determine the quality of the training set and 2) provide the relation between the test performance and the proposed measures.The measures are derived from pairwise affinities between training exemplars of the positive class and they have a generative nature. We show that the performance of the state of the art methods, on the test set, can be reasonably predicted based on the values of the proposed measures on the training set.These measures open up a wide range of applications to the recognition community enabling us to analyze the behavior of the learning algorithms w.r.t the properties of the training data. This will in turn enable us to devise rules for the automatic selection of training data that maximize the quantified quality of the training set and thereby improve recognition performance.

show abstract

Section: Quantifying the Quality Of A Training Setmentioning

confidence: 99%

Section: Quantifying the Quality Of A Training Setmentioning

confidence: 99%

Section: Quantifying the Quality Of A Training Setmentioning

confidence: 99%

Section: Quantifying the Quality Of A Training Setmentioning

confidence: 99%

Section: Quantifying the Quality Of A Training Setmentioning

confidence: 99%

See 3 more Smart Citations

Properties of Datasets Predict the Performance of Classifiers

Aghazadeh

Carlsson

2013

Procedings of the British Machine Vision Conference 2013

View full text Add to dashboard Cite

show abstract

Weighted k‐nearest neighbor based data complexity metrics for imbalanced datasets

Singh

Gosain

Saha

2020

Statistical Analysis

View full text Add to dashboard Cite

Empirical behavior of a classifier depends strongly on the characteristics of the underlying imbalanced dataset; therefore, an analysis of intrinsic data complexity would appear to be vital in order to choose classifiers suitable for particular problems. Data complexity metrics (CMs), a fairly recent proposal, identify dataset features which imply some difficulty for the classification task and identify relationships with classifier accuracy. In this paper, we introduce two CMs for imbalanced datasets, which help in explaining the factors responsible for the deterioration in classifier performance. These metrics are based on the weighted k‐nearest neighbors approach. The experiments are performed in MATLAB software using 48 simulated datasets and 22 real‐world datasets for different choices of neighborhood size k considered as 3, 5, 7, 9, 11. The results help to illustrate the usefulness of the proposed metrics.

show abstract