A systematic analysis of performance measures for classification tasks

Sokolova, Marina; Lapalme, Guy

doi:10.1016/j.ipm.2009.03.002

Cited by 4,273 publications

(2,371 citation statements)

References 17 publications

Supporting

Mentioning

2,081

Contrasting

Unclassified

Order By: Relevance

“…However, these values hide disparities between classes. The F-score, e.g., a synthetic accuracy metric, is used here to compare the classification performance at the class level [59].…”

Section: Land-use Classification Using Random Forestmentioning

confidence: 99%

Mapping Urban Land Use at Street Block Level Using OpenStreetMap, Remote Sensing Data, and Spatial Metrics

Grippa

Georganos

Zarougui

et al. 2018

IJGI

View full text Add to dashboard Cite

Up-to-date and reliable land-use information is essential for a variety of applications such as planning or monitoring of the urban environment. This research presents a workflow for mapping urban land use at the street block level, with a focus on residential use, using very-high resolution satellite imagery and derived land-cover maps as input. We develop a processing chain for the automated creation of street block polygons from OpenStreetMap and ancillary data. Spatial metrics and other street block features are computed, followed by feature selection that reduces the initial datasets by more than 80%, providing a parsimonious, discriminative, and redundancy-free set of features. A random forest (RF) classifier is used for the classification of street blocks, which results in accuracies of 84% and 79% for five and six land-use classes, respectively. We exploit the probabilistic output of RF to identify and relabel blocks that have a high degree of uncertainty. Finally, the thematic precision of the residential blocks is refined according to the proportion of the built-up area. The output data and processing chains are made freely available. The proposed framework is able to process large datasets, given that the cities in the case studies, Dakar and Ouagadougou, cover more than 1000 km 2 in total, with a spatial resolution of 0.5 m.

show abstract

“…However, these values hide disparities between classes. The F-score, e.g., a synthetic accuracy metric, is used here to compare the classification performance at the class level [59].…”

Section: Land-use Classification Using Random Forestmentioning

confidence: 99%

Mapping Urban Land Use at Street Block Level Using OpenStreetMap, Remote Sensing Data, and Spatial Metrics

Grippa

Georganos

Zarougui

et al. 2018

IJGI

View full text Add to dashboard Cite

show abstract

“…Basically, the set of traces classified by the model (to determine whether it accepts them as positive or rejects them as negative), and from this we end up with four numbers: The number of True Positives (TP), True Negatives (TN), False Positives (FP) and False Negatives (FN). From these numbers it is possible to calculate various 'Binary Classification Measures' [41].…”

Section: Measuring Accuracymentioning

confidence: 99%

Inferring extended finite state machine models from software executions

2015

View full text Add to dashboard Cite

The ability to reverse-engineer models of software behaviour is valuable for a wide range of software maintenance, validation and verification tasks. Current reverse-engineering techniques focus either on control-specific behaviour (e.g., in the form of Finite State Machines), or data-specific behaviour (e.g., as pre / post-conditions or invariants). However, typical software behaviour is usually a product of the two; models must combine both aspects to fully represent the software's operation. Extended Finite State Machines (EFSMs) provide such a model. Although attempts have been made to infer EFSMs, these have been problematic. The models inferred by these techniques can be non-deterministic, the inference algorithms can be inflexible, and only applicable to traces with specific characteristics. This paper presentsWe thank Ivo Krka at the University of Southern California for providing us with some of the traces for the evaluation. Ramsay Taylor and John Derrick are supported by the EU FP7 PROWESS projects. Neil Walkinshaw is grateful for the support provided by the UK Ministry of Defence (MOD) through the BATS and HASTE projects. Information contained in this document should not be interpreted as representing the views of the MOD, nor should it be assumed that it reflects any current or future MOD policy. The information cannot supersede any statutory or contractual requirements or liabilities and is offered without prejudice or commitment.

show abstract

“…Classification accuracy (AC) (Sokolova & Lapalme, 2009): percentage of correctly classified samples (or sample sequences for the HMM classifier). 2.…”

Section: Model Validationmentioning

confidence: 99%

“…2. F1-score (FS) (Sokolova & Lapalme, 2009): a measure of how well the classifier was able to distinguish between classes given an unbalanced dataset. 3.…”

Section: Model Validationmentioning

confidence: 99%

Analysis of Drivers' Head and Eye Movement Correspondence: Predicting Drivers' Glance Location Using Head Rotation Data

Muñoz

Lee

Reimer

et al. 2015

Proceedings of the 8th International Driving Symposium on Human Factors in Driver Assessment, Training, and Vehicle Design: Dri

View full text Add to dashboard Cite

Summary:The relationship between a driver's glance pattern and corresponding head rotation is not clearly defined. Head rotation and eye glance data drawn from a study conducted by the Virginia Tech Transportation Institute in support of methods development for the Strategic Highway Research Program (SHRP 2) naturalistic driving study were assessed. The data were utilized as input to classifiers that predicted glance allocation to the road and the center stack. A predictive accuracy of 83% was achieved with Hidden Markov Models. Results suggest that although there are individual differences in head-eye correspondence while driving, head-rotation data may be a useful predictor of glance location. Future work needs to investigate the correspondence across a wider range of individuals, traffic conditions, secondary tasks, and areas of interest.

show abstract

A systematic analysis of performance measures for classification tasks

Cited by 4,273 publications

References 17 publications

Mapping Urban Land Use at Street Block Level Using OpenStreetMap, Remote Sensing Data, and Spatial Metrics

Mapping Urban Land Use at Street Block Level Using OpenStreetMap, Remote Sensing Data, and Spatial Metrics

Inferring extended finite state machine models from software executions

Analysis of Drivers' Head and Eye Movement Correspondence: Predicting Drivers' Glance Location Using Head Rotation Data

Contact Info

Product

Resources

About