Metrics for Evaluating Classification Algorithms

Muntean, Mihaela; Militaru, Florin-Daniel

doi:10.1007/978-981-19-6755-9_24

Cited by 7 publications

(2 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Accuracy indicates overall correctness, sensitivity measures true positive rate, precision assesses positive prediction accuracy, and F1-score balances both precision and sensitivity through harmonic mean [38]. These metrics are prevalent in classification performance evaluation [39][40][41][42], where higher values of AUC denote model's superior discrimination ability between positive and negative classes. A two-tailed P-value of less than 0.05 indicated statistical significance.…”

Section: Other Statistical Methodsmentioning

confidence: 99%

Development and application of a deep learning-based comprehensive early diagnostic model for chronic obstructive pulmonary disease

Zhu,

Zhao,

et al. 2024

Respir Res

View full text Add to dashboard Cite

Background Chronic obstructive pulmonary disease (COPD) is a frequently diagnosed yet treatable condition, provided it is identified early and managed effectively. This study aims to develop an advanced COPD diagnostic model by integrating deep learning and radiomics features. Methods We utilized a dataset comprising CT images from 2,983 participants, of which 2,317 participants also provided epidemiological data through questionnaires. Deep learning features were extracted using a Variational Autoencoder, and radiomics features were obtained using the PyRadiomics package. Multi-Layer Perceptrons were used to construct models based on deep learning and radiomics features independently, as well as a fusion model integrating both. Subsequently, epidemiological questionnaire data were incorporated to establish a more comprehensive model. The diagnostic performance of standalone models, the fusion model and the comprehensive model was evaluated and compared using metrics including accuracy, precision, recall, F1-score, Brier score, receiver operating characteristic curves, and area under the curve (AUC). Results The fusion model exhibited outstanding performance with an AUC of 0.952, surpassing the standalone models based solely on deep learning features (AUC = 0.844) or radiomics features (AUC = 0.944). Notably, the comprehensive model, incorporating deep learning features, radiomics features, and questionnaire variables demonstrated the highest diagnostic performance among all models, yielding an AUC of 0.971. Conclusion We developed and implemented a data fusion strategy to construct a state-of-the-art COPD diagnostic model integrating deep learning features, radiomics features, and questionnaire variables. Our data fusion strategy proved effective, and the model can be easily deployed in clinical settings. Trial registration Not applicable. This study is NOT a clinical trial, it does not report the results of a health care intervention on human participants.

show abstract

Section: Other Statistical Methodsmentioning

confidence: 99%

Development and application of a deep learning-based comprehensive early diagnostic model for chronic obstructive pulmonary disease

Zhu,

Zhao,

et al. 2024

Respir Res

View full text Add to dashboard Cite

show abstract

“…The latter can also lead to overfitting (Figure 6), where a real-world test of an arguably perfect model may fail. Thus, keeping in mind the practical applications, potential biases, ethical implications, and general application of the model are more tangible criteria in performance evaluation (Duffull and Isbister, 2022;Muntean and Militaru, 2023).…”

Section: Machine Learning Model Representationmentioning

confidence: 99%

Machine learning in toxicological sciences: opportunities for assessing drug toxicity

Tonoyan,

Siraki

2024

Front. Drug Discov.

View full text Add to dashboard Cite

Machine learning (ML) in toxicological sciences is growing exponentially, which presents unprecedented opportunities and brings up important considerations for using ML in this field. This review discusses supervised, unsupervised, and reinforcement learning and their applications to toxicology. The application of the scientific method is central to the development of a ML model. These steps involve defining the ML problem, constructing the dataset, transforming the data and feature selection, choosing and training a ML model, validation, and prediction. The need for rigorous models is becoming more of a requirement due to the vast number of chemicals and their interaction with biota. Large datasets make this task possible, though selecting databases with overlapping chemical spaces, amongst other things, is an important consideration. Predicting toxicity through machine learning can have significant societal impacts, including enhancements in assessing risks, determining clinical toxicities, evaluating carcinogenic properties, and detecting harmful side effects of medications. We provide a concise overview of the current state of this topic, focusing on the potential benefits and challenges related to the availability of extensive datasets, the methodologies for analyzing these datasets, and the ethical implications involved in applying such models.

show abstract

Prediction of student exam performance using data mining classification algorithms

Khairy,

Alharbi,

Amasha

et al. 2024

Educ Inf Technol

View full text Add to dashboard Cite

Student outcomes are of great importance in higher education institutions. Accreditation bodies focus on them as an indicator to measure the performance and effectiveness of the institution. Forecasting students’ academic performance is crucial for every educational establishment seeking to enhance performance and perseverance of its students and reduce the failure rate in the future. The main goal of this study is to predict the performance of undergraduate first-level students in the Computer Department during the years 2016 to 2021 to enhance their performance in future by discovering the best algorithm use to analyze the educational data to identify the students’ academic performance. The secondary data was collected by reviewing the Student Affairs Department at the Faculty of Specific Education at Damietta University, in addition to the Statistics Department at the university. The dataset contained 830 instances after excluding 139 instances of missing values, irrelevant rows, and outliers. The dataset was divided into train (577 instances (70%)), test (253 instances (30%)) and involved six features such year, midterm, practical exam, writing exam, final total degree, and grade. This paper use five machine learning (ML) algorithms which was selected according to the literature review and high accuracy in predicting educational data mining: For the purpose of comparison, a number of different machine learning algorithms, such as Random Forest, Decision Tree, Naive Bayes, Neural Network, and K-Nearest Neighbours, were utilized and evaluated with evaluation metrics such as confusion matrix, accuracy, precision, recall, and F-measure. The Random Forest and Decision Tree classifiers emerged as the top-performing algorithms, accurately categorizing 250 instances when predicting students' performance in the statistics course. This was determined based on the findings of the study. Out of a total of 253 instances that were included in the testing set, they only made three incorrect classifications.

show abstract

Metrics for Evaluating Classification Algorithms

Cited by 7 publications

References 17 publications

Development and application of a deep learning-based comprehensive early diagnostic model for chronic obstructive pulmonary disease

Development and application of a deep learning-based comprehensive early diagnostic model for chronic obstructive pulmonary disease

Machine learning in toxicological sciences: opportunities for assessing drug toxicity

Prediction of student exam performance using data mining classification algorithms

Contact Info

Product

Resources

About