AUC-Based Extreme Learning Machines for Supervised and Semi-Supervised Imbalanced Classification

Wang, Guanjin; Wong, Kok Wai; Lu, Jie

doi:10.1109/tsmc.2020.2982226

Cited by 34 publications

(5 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To measure the performance of the proposed model, several evaluation metrics were used such as precision ( Miao & Zhu, 2021 ), recall ( Tharwat, 2020 ), F-measure ( Soleymani, Granger & Fumera, 2020 ), accuracy ( Dinga et al, 2019 ), and Area Under Curve (AUC) ( Wang, Wong & Lu, 2020 ; Kabir & Ludwig, 2019b ). These were derived from the confusion matrix ( Xu, Zhang & Miao, 2020 ; Markoulidakis et al, 2021 ) to calculate different evaluations for the proposed model.…”

Section: Methodsmentioning

confidence: 99%

Swarm intelligence-based model for improving prediction performance of low-expectation teams in educational software engineering projects

Al‐Ahmad

Al-Zoubi

Kabir

et al. 2022

PeerJ Computer Science

View full text Add to dashboard Cite

Software engineering is one of the most significant areas, which extensively used in educational and industrial fields. Software engineering education plays an essential role in keeping students up to date with software technologies, products, and processes that are commonly applied in the software industry. The software development project is one of the most important parts of the software engineering course, because it covers the practical side of the course. This type of project helps strengthening students’ skills to collaborate in a team spirit to work on software projects. Software project involves the composition of software product and process parts. Software product part represents software deliverables at each phase of Software Development Life Cycle (SDLC) while software process part captures team activities and behaviors during SDLC. The low-expectation teams face challenges during different stages of software project. Consequently, predicting performance of such teams is one of the most important tasks for learning process in software engineering education. The early prediction of performance for low-expectation teams would help instructors to address difficulties and challenges related to such teams at earliest possible phases of software project to avoid project failure. Several studies attempted to early predict the performance for low-expectation teams at different phases of SDLC. This study introduces swarm intelligence -based model which essentially aims to improve the prediction performance for low-expectation teams at earliest possible phases of SDLC by implementing Particle Swarm Optimization-K Nearest Neighbours (PSO-KNN), and it attempts to reduce the number of selected software product and process features to reach higher accuracy with identifying less than 40 relevant features. Experiments were conducted on the Software Engineering Team Assessment and Prediction (SETAP) project dataset. The proposed model was compared with the related studies and the state-of-the-art Machine Learning (ML) classifiers: Sequential Minimal Optimization (SMO), Simple Linear Regression (SLR), Naïve Bayes (NB), Multilayer Perceptron (MLP), standard KNN, and J48. The proposed model provides superior results compared to the traditional ML classifiers and state-of-the-art studies in the investigated phases of software product and process development.

show abstract

Section: Methodsmentioning

confidence: 99%

Swarm intelligence-based model for improving prediction performance of low-expectation teams in educational software engineering projects

Al‐Ahmad

Al-Zoubi

Kabir

et al. 2022

PeerJ Computer Science

View full text Add to dashboard Cite

show abstract

“…According to [54], a high AUC value results in a model robust against class imbalance. Furthermore, the F1 score can work well when data are unbalanced since it represents the harmonic mean of precision and TPR.…”

Section: Metrics Used To Evaluate Classification Performancementioning

confidence: 99%

Unbalanced Web Phishing Classification through Deep Reinforcement Learning

Maci,

Santorsola,

Coscia

et al. 2023

Computers

View full text Add to dashboard Cite

Web phishing is a form of cybercrime aimed at tricking people into visiting malicious URLs to exfiltrate sensitive data. Since the structure of a malicious URL evolves over time, phishing detection mechanisms that can adapt to such variations are paramount. Furthermore, web phishing detection is an unbalanced classification task, as legitimate URLs outnumber malicious ones in real-life cases. Deep learning (DL) has emerged as a promising technique to minimize concept drift to enhance web phishing detection. Deep reinforcement learning (DRL) combines DL with reinforcement learning (RL); that is, a sequential decision-making paradigm in which the problem to be addressed is expressed as a Markov decision process (MDP). Recent studies have proposed an ad hoc MDP formulation to tackle unbalanced classification tasks called the imbalanced classification Markov decision process (ICMDP). In this paper, we exploit the ICMDP to present a double deep Q-Network (DDQN)-based classifier to address the unbalanced web phishing classification problem. The proposed algorithm is evaluated on a Mendeley web phishing dataset, from which three different data imbalance scenarios are generated. Despite a significant training time, it results in better geometric mean, index of balanced accuracy, F1 score, and area under the ROC curve than other DL-based classifiers combined with data-level sampling techniques in all test cases.

show abstract

“…Taking into consideration the comparative analysis from Table 2 for supervised machine learning classifiers, four machine learning classifiers viz. Random Forest (RF), Support Vector Machine(SVM), Naïve Bayes (NB) and Logistic regression(LR) [13][14][15][16][17] were used for supervised classification using Machine Learning [2,18,19] on the above datasets. The train test method with Stratified Crossfold with k=10 strategy was used for classifier experimentation.…”

Section: Datasets Usedmentioning

confidence: 99%

Healthcare Critical Diagnosis Accuracy

Pankaj Javale,

Desai

2023

Int. j. electr. comput. eng. syst. (Online)

View full text Add to dashboard Cite

Since at least a decade, Machine Learning has attracted the interest of researchers. Among the topics of discussion is the application of Machine Learning (ML) and Deep Learning (DL) to the healthcare industry. Several implementations are performed on the medical dataset to verify its precision. The four main players, True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN), play a crucial role in determining the classifier's performance. Various metrics are provided based on the main players. Selecting the appropriate performance metric is a crucial step. In addition to TP and TN, FN should be given greater weight when a healthcare dataset is evaluated for disease diagnosis or detection. Thus, a suitable performance metric must be considered. In this paper, a novel machine learning metric referred to as Healthcare-Critical-Diagnostic-Accuracy (HCDA) is proposed and compared to the well-known metrics accuracy and ROC_AUC score. The machine learning classifiers Support Vector Machine (SVM), Logistic Regression (LR), Random Forest (RF), and Naive Bayes (NB) are implemented on four distinct datasets. The obtained results indicate that the proposed HCDA metric is more sensitive to FN counts. The results show, that even if there is rise in %FN for dataset 1 to 10.31 % then too accuracy is 83% ad HCDA shows correlated drop to 72.70 %. Similarly, in dataset 2 if %FN rises to 14.80 for LR classifier, accuracy is 78.2 % and HCDA is 63.45 %. Similar kind of results are obtained for dataset 3 and 4 too. More FN counts result in a lower HCDA score, and vice versa. In common exiting metrics such as Accuracy and ROC_AUC score, even as the FN count increases, the score increases, which is misleading. As a result, it can be concluded that the proposed HCDA is a more robust and accurate metric for Critical Healthcare Analysis, as FN conditions for disease diagnosis and detection are taken into account more than TP and TN.

show abstract

AUC-Based Extreme Learning Machines for Supervised and Semi-Supervised Imbalanced Classification

Cited by 34 publications

References 47 publications

Swarm intelligence-based model for improving prediction performance of low-expectation teams in educational software engineering projects

Swarm intelligence-based model for improving prediction performance of low-expectation teams in educational software engineering projects

Unbalanced Web Phishing Classification through Deep Reinforcement Learning

Healthcare Critical Diagnosis Accuracy

Contact Info

Product

Resources

About