Enhancing Confusion Entropy (CEN) for binary and multiclass classification

Delgado, Rosario; Núñez-González, J. David

doi:10.1371/journal.pone.0210264

Cited by 20 publications

(8 citation statements)

References 14 publications

(28 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the 2010s, several alternative novel measures have been proposed, either to tackle a particular issue such as imbalance [34, 77], or with a broader purpose. Among them, we mention the confusion entropy [78, 79], a statistical score comparable with MCC [80], and the K measure [81], a theoretically grounded measure that relies on a strong axiomatic base.…”

Section: Introductionmentioning

confidence: 99%

The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation

2020

View full text Add to dashboard Cite

BackgroundTo evaluate binary classifications and their confusion matrices, scientific researchers can employ several statistical rates, accordingly to the goal of the experiment they are investigating. Despite being a crucial issue in machine learning, no widespread consensus has been reached on a unified elective chosen measure yet. Accuracy and F1 score computed on confusion matrices have been (and still are) among the most popular adopted metrics in binary classification tasks. However, these statistical measures can dangerously show overoptimistic inflated results, especially on imbalanced datasets.ResultsThe Matthews correlation coefficient (MCC), instead, is a more reliable statistical rate which produces a high score only if the prediction obtained good results in all of the four confusion matrix categories (true positives, false negatives, true negatives, and false positives), proportionally both to the size of positive elements and the size of negative elements in the dataset.ConclusionsIn this article, we show how MCC produces a more informative and truthful score in evaluating binary classifications than accuracy and F1 score, by first explaining the mathematical properties, and then the asset of MCC in six synthetic use cases and in a real genomics scenario. We believe that the Matthews correlation coefficient should be preferred to accuracy and F1 score in evaluating binary classification tasks by all scientific communities.

show abstract

Section: Introductionmentioning

confidence: 99%

The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation

2020

View full text Add to dashboard Cite

show abstract

“…The true positive rate (TPR), positive predictive value (PPV), true negative rate (TNR), and modified confusion entropy (MCEN) were also measured for these three FCNNs using 10 000 test images. [39] Figure 4b,c shows that the values of the TPR and PPV of BP and DGC-S were different from those of DGC-O for some labels due to convergence at different weight values. However, as shown in Figure 4d, all three methods exhibit similar TNRs.…”

Section: Trainability Validationmentioning

confidence: 95%

Direct Gradient Calculation: Simple and Variation‐Tolerant On‐Chip Training Method for Neural Networks

Kim

Hwang

Kwon

et al. 2021

Advanced Intelligent Systems

View full text Add to dashboard Cite

On‐chip training of neural networks (NNs) is regarded as a promising training method for neuromorphic systems with analog synaptic devices. Herein, a novel on‐chip training method called direct gradient calculation (DGC) is proposed to substitute conventional backpropagation (BP). In this method, the gradients of a cost function with respect to the weights are calculated directly by sequentially applying a small temporal change to each weight and then measuring the change in cost value. DGC achieves a similar accuracy to that of BP while performing a handwritten digit classification task, validating its training feasibility. In particular, DGC can be applied to analog hardware‐based convolutional NNs (CNNs), which is considered to be a challenging task, enabling appropriate on‐chip training. A hybrid method is also proposed that efficiently combines DGC and BP for training CNNs, and the method achieves a similar accuracy to that of BP and DGC while enhancing the training speed. Furthermore, networks utilizing DGC maintain a higher level of accuracy than those using BP in the presence of variations in hardware (such as synaptic device conductance and neuron circuit component variations) while requiring fewer circuit components.

show abstract

“…There are papers focusing on properties of a particular measure, for instance, Cohen's Kappa [8,22], Confusion Entropy [7], or Balanced Accuracy [2]. Some papers go beyond the threshold measures considered in our paper.…”

Section: A Related Workmentioning

confidence: 99%

Good Classification Measures and How to Find Them

Gösgens¹,

Zhiyanov²,

Tikhonov³

et al. 2022

Preprint

View full text Add to dashboard Cite

Several performance measures can be used for evaluating classification results: accuracy, F-measure, and many others. Can we say that some of them are better than others, or, ideally, choose one measure that is best in all situations? To answer this question, we conduct a systematic analysis of classification performance measures: we formally define a list of desirable properties and theoretically analyze which measures satisfy which properties. We also prove an impossibility theorem: some desirable properties cannot be simultaneously satisfied. Finally, we propose a new family of measures satisfying all desirable properties except one. This family includes the Matthews Correlation Coefficient and a so-called Symmetric Balanced Accuracy that was not previously used in classification literature. We believe that our systematic approach gives an important tool to practitioners for adequately evaluating classification results. 1 We describe related research in detail in Appendix A.35th Conference on Neural Information Processing Systems (NeurIPS 2021).

show abstract

Enhancing Confusion Entropy (CEN) for binary and multiclass classification

Cited by 20 publications

References 14 publications

The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation

The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation

Direct Gradient Calculation: Simple and Variation‐Tolerant On‐Chip Training Method for Neural Networks

Good Classification Measures and How to Find Them

Contact Info

Product

Resources

About