LRID: A new metric of multi-class imbalance degree based on likelihood-ratio test

Zhu, Rui; Wang, Ziyu; Ma, Zhanyu; Wang, Guijin; Xue, Jing-Hao

doi:10.1016/j.patrec.2018.09.012

Cited by 22 publications

(22 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a future work, we plan to study the problem on a domain-specific level by applying tuning based extracted features from malware samples to see the positive effects imposed on the classification problem. We also want to study the imbalance factor and data factors that may cause the performance degradation and its relation to the domain by calculating the Imbalance Degree (ID) and Likelihood Ratio (LRID) presented in [34] and [35].…”

Section: Discussionmentioning

confidence: 99%

Comparative Analysis on Imbalanced Multi-class Classification for Malware Samples using CNN

Alzammam

Binsalleeh

AsSadhan

et al. 2020

2019 International Conference on Advances in the Emerging Computing Technologies (AECT)

View full text Add to dashboard Cite

Malware considered as one of the main actors in cyber attacks. Everyday, the number of unique malware samples are in the rise, however the ratio of benign software still greatly outnumbers malware samples. In machine learning, such datasets are known as imbalanced, where the majority class label greatly dominate the other ones. In this paper, we present a comparative analysis and evaluation of some of the proposed techniques in the literature to address the problem of classifying imbalanced multiclass malware datasets. We used Convolutional Neural Network (CNN) as a classification algorithm to study the effect of imbalanced datasets on deep learning approaches. The experiments are conducted on three publicly available imbalanced datasets. Our performance analysis shows that methods such as cost sensitive learning, oversampling and cross validation have positive effects on the model classification performance with varying degree. While others like using pre-trained models require more special parameter settings. However, best practice may change according to the problem domain.

show abstract

Section: Discussionmentioning

confidence: 99%

Comparative Analysis on Imbalanced Multi-class Classification for Malware Samples using CNN

Alzammam

Binsalleeh

AsSadhan

et al. 2020

2019 International Conference on Advances in the Emerging Computing Technologies (AECT)

View full text Add to dashboard Cite

show abstract

“…Following [6] and [7] , we calculate both the Spearman rank correlation coe cient (SRCC) and the Pearson correlation coe cient (PCC) to assess the correlation between the adjusted IR and the F1 score. For simulation 1, we combine the F1 scores and the adjusted IRs for datasets generated in situations S2, S3 and S4 with fixed IR and calculate the correlations between them.…”

Section: Simulated Datamentioning

confidence: 99%

“…However, the penalty term involving the number of minority classes makes ID not well correlated with the classification performance. This is because it is not always true that the class-imbalance extent is higher for the data with more minority classes [7]. Zhu et al [7] propose a new metric, the likelihood-ratio imbalance degree (LRID), which is based on the likelihood ratio test.…”

Section: Introductionmentioning

confidence: 99%

“…This is because it is not always true that the class-imbalance extent is higher for the data with more minority classes [7]. Zhu et al [7] propose a new metric, the likelihood-ratio imbalance degree (LRID), which is based on the likelihood ratio test. It is a high-resolution metric and is negatively correlated with the classification performance.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Adjusting the imbalance ratio by the dimensionality of imbalanced data

Zhu

Guo²,

Xue

2020

Pattern Recognition Letters

Self Cite

View full text Add to dashboard Cite

Class-imbalance extent metrics measure how imbalanced the data are. In pattern classification, it is usually expected that the higher the imbalance extent, the worse the classification performance, and thus an appropriate imbalance extent metric should show a negative correlation with the classification performance. Existing metrics, such as the popular imbalance ratio (IR), only consider the e↵ect of the sample sizes of di↵erent classes. However, we note that the dimensionality of imbalanced data also a↵ects the classification performance. Datasets with the same IR can present distinct classification performances when their dimensionalities are di↵erent, making IR suboptimal to reflect the imbalance extent for classification. We also observe that the classification performance becomes better with more discriminative features. Inspired by these observations, we propose a new imbalance extent metric, the adjusted IR, by adding a penalty term of the number of discriminative features that is e↵ectively determined by the Pearson correlation test. The adjusted IR adaptively revises the IR when the number of discriminative features varies. The empirical studies demonstrate the e↵ectiveness of the adjusted IR, in terms of its better negative correlation with the classification performance.

show abstract

“…Imbalance ratio (IR) is the widely accepted measure to determine imbalance data. In Equation (1), IR is the ratio of the number of records of the majority class between the number of records of minority class [34]. A dataset can be considered imbalanced if IR > 1.5 [35].…”

Section: Imbalance Ratiomentioning

confidence: 99%

Classification of Guillain–Barré Syndrome Subtypes Using Sampling Techniques with Binary Approach

et al. 2020

View full text Add to dashboard Cite

Guillain–Barré Syndrome (GBS) is an unusual disorder where the body’s immune system affects the peripheral nervous system. GBS has four main subtypes, whose treatments vary among them. Severe cases of GBS can be fatal. This work aimed to investigate whether balancing an original GBS dataset improves the predictive models created in a previous study. purpleBalancing a dataset is to pursue symmetry in the number of instances of each of the classes.The dataset includes 129 records of Mexican patients diagnosed with some subtype of GBS. We created 10 binary datasets from the original dataset. Then, we balanced these datasets using four different methods to undersample the majority class and one method to oversample the minority class. Finally, we used three classifiers with different approaches to creating predictive models. The results show that balancing the original dataset improves the previous predictive models. The goal of the predictive models is to identify the GBS subtypes applying Machine Learning algorithms. It is expected that specialists may use the model to have a complementary diagnostic using a reduced set of relevant features. Early identification of the subtype will allow starting with the appropriate treatment for patient recovery. This is a contribution to exploring the performance of balancing techniques with real data.

show abstract

LRID: A new metric of multi-class imbalance degree based on likelihood-ratio test

Cited by 22 publications

References 14 publications

Comparative Analysis on Imbalanced Multi-class Classification for Malware Samples using CNN

Comparative Analysis on Imbalanced Multi-class Classification for Malware Samples using CNN

Adjusting the imbalance ratio by the dimensionality of imbalanced data

Classification of Guillain–Barré Syndrome Subtypes Using Sampling Techniques with Binary Approach

Contact Info

Product

Resources

About