On the performance of Matthews correlation coefficient (MCC) for imbalanced dataset

Zhu, Qiuming

doi:10.1016/j.patrec.2020.03.030

Cited by 128 publications

(73 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The ROC metric helps in discriminating between classes and is found effective for medical diagnostic evaluation (Swets, 1986, Tilaki, 2013. MCC is seen to be effective if the classes are balanced and deteriorate if they are unbalanced since it gets unevenly distributed [Zhu, (2020)]. Table 3 gives the comparison with other data mining methods.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

A Modified- Weighted- K - Nearest Neighbour and Cuckoo Search Hybrid Model for Breast Cancer Classification

Mathew¹,

Kumar²

2021

INDJCSE

View full text Add to dashboard Cite

One of the leading death-causing cancers in women is Breast Cancer. Accurate, precise, and early diagnosis is a crucial solution to survival. Data mining techniques have proved to produce good results in disease diagnosis. Feature search techniques are useful in identifying the relevant features for classification thus reducing time and effort. Class inequality is a significant challenge and one of the methods to overcome it is class balancing. In certain cases, the negative class is the majority class. To be specific; the negative class has a more number of instances than the positive class, so the overall classifier performance may be high; consequently, the classifier performance in accurately identifying positive instances gets overlooked. In this paper, a combination of two class balancing approaches is applied. It is used to balance the number of instances in each of the target classes. k-Nearest Neighbour classifier is a simple, easy to implement, and robust classifier with few parameters needed to be tuned. In this paper, we propose a k-Nearest Neighbour Classifier model implemented with feature search using Cuckoo search and Class balancing to classify Breast Cancer. The proposed model produced an accuracy of 99.41 %., ROC of 0.999, and MCC of 0.988.

show abstract

Section: Resultsmentioning

confidence: 99%

“…MCC gives a high score only if good prediction results are achieved in all four categories of the confusion matrix -(TP, TN, FP, FN). MCC is seen to be effective if the classes are balanced and deteriorate if they are unbalanced since it gets unevenly distributed[Zhu, (2020)]. Table3gives the comparison with other data mining methods.…”

mentioning

confidence: 99%

A Modified- Weighted- K - Nearest Neighbour and Cuckoo Search Hybrid Model for Breast Cancer Classification

Mathew¹,

Kumar²

2021

INDJCSE

View full text Add to dashboard Cite

show abstract

“…As far as skewness is concerned, our L FPL handles it automatically by assigning the class sample size ratios of √B/A and √A/B to TN and TP, respectively, during MCC calculation, where A and B specify the majority and minority classes, respectively [ 36 ]. That is if A = TN + FP and B = TP + FN then FP = A − TN and FN = B − TP, so, Equation (10) becomes: MCC = (−√AB + √B/A TN + √A/B TP + ε)/√ ((TP + A − TN) (TN + B − TP) + ε) …”

Section: Methodsmentioning

confidence: 99%

A Novel Focal Phi Loss for Power Line Segmentation with Auxiliary Classifier U-Net

Jaffari

Hashmani

Reyes-Aldasoro

2021

Sensors

View full text Add to dashboard Cite

The segmentation of power lines (PLs) from aerial images is a crucial task for the safe navigation of unmanned aerial vehicles (UAVs) operating at low altitudes. Despite the advances in deep learning-based approaches for PL segmentation, these models are still vulnerable to the class imbalance present in the data. The PLs occupy only a minimal portion (1–5%) of the aerial images as compared to the background region (95–99%). Generally, this class imbalance problem is addressed via the use of PL-specific detectors in conjunction with the popular class balanced cross entropy (BBCE) loss function. However, these PL-specific detectors do not work outside their application areas and a BBCE loss requires hyperparameter tuning for class-wise weights, which is not trivial. Moreover, the BBCE loss results in low dice scores and precision values and thus, fails to achieve an optimal trade-off between dice scores, model accuracy, and precision–recall values. In this work, we propose a generalized focal loss function based on the Matthews correlation coefficient (MCC) or the Phi coefficient to address the class imbalance problem in PL segmentation while utilizing a generic deep segmentation architecture. We evaluate our loss function by improving the vanilla U-Net model with an additional convolutional auxiliary classifier head (ACU-Net) for better learning and faster model convergence. The evaluation of two PL datasets, namely the Mendeley Power Line Dataset and the Power Line Dataset of Urban Scenes (PLDU), where PLs occupy around 1% and 2% of the aerial images area, respectively, reveal that our proposed loss function outperforms the popular BBCE loss by 16% in PL dice scores on both the datasets, 19% in precision and false detection rate (FDR) values for the Mendeley PL dataset and 15% in precision and FDR values for the PLDU with a minor degradation in the accuracy and recall values. Moreover, our proposed ACU-Net outperforms the baseline vanilla U-Net for the characteristic evaluation parameters in the range of 1–10% for both the PL datasets. Thus, our proposed loss function with ACU-Net achieves an optimal trade-off for the characteristic evaluation parameters without any bells and whistles. Our code is available at Github.

show abstract

“…Due to its symmetry, the MCC metric provides a good indicator of classification performance even for highly unbalanced datasets (as is the case here for very small temporal windows). However, MCC can still display biases due to class imbalances (Zhu, 2020). To address this limitation and improve comparisons of classification performance across window sizes with different inherent class imbalances, we introduce a slightly adjusted version of MCC: where K is the minimum MCC achieved across all classifications with the same class imbalance(i.e.…”

Section: Methodsmentioning

confidence: 99%

Generalizable cursor click control using grasp-related neural transients

Dekleva

Weiss

Boninger

et al. 2020

Preprint

View full text Add to dashboard Cite

Intracortical brain-computer interfaces (iBCI) have the potential to restore independence for individuals with significant motor or communication impairments. One of the most realistic avenues for clinical translation of iBCI technology is to enable control of a computer cursor−i.e. movement-related neural activity is interpreted (decoded) and used to drive cursor function. Both nonhuman primate and human studies have demonstrated high-level cursor translation control using attempted upper limb reaching movements. However, cursor click control−based on identifying attempted grasp−has only been successful in providing discrete click functionality; the ability to maintain click during translation does not yet exist. Here we present a novel decoding approach for cursor click based on identifying transient neural responses that emerge at the onset and offset of intended hand grasp. We demonstrate in a human participant, who used the BCI system independently in his home, that this transient-based approach provides high-functioning, generalized click control that can be used for both point-and-click and click-and-drag applications.

show abstract

On the performance of Matthews correlation coefficient (MCC) for imbalanced dataset

Cited by 128 publications

References 6 publications

A Modified- Weighted- K - Nearest Neighbour and Cuckoo Search Hybrid Model for Breast Cancer Classification

A Modified- Weighted- K - Nearest Neighbour and Cuckoo Search Hybrid Model for Breast Cancer Classification

A Novel Focal Phi Loss for Power Line Segmentation with Auxiliary Classifier U-Net

Generalizable cursor click control using grasp-related neural transients

Contact Info

Product

Resources

About