Must-have Qualities of Clinical Research on Artificial Intelligence and Machine Learning

Koçak, Burak; Cuocolo, Renato; Santos, Daniel Pinto dos; Stanzione, Arnaldo; Ugga, Lorenzo

doi:10.4274/balkanmedj.galenos.2022.2022-11-51

Cited by 25 publications

(17 citation statements)

References 106 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Predictive model evaluation especially in healthcare and other real-world application systems must take into account the peculiarity of its dataset especially when assessing predictive model performance [8]. General prediction accuracy score show results obtained from both observed and predicted values.…”

Section: Balanced Accuracy Process Diagrammentioning

confidence: 99%

A systematic review of prediction accuracy as an evaluation measure for determining machine learning model performance in healthcare systems

Owusu-Adjei

Hayfron-Acquah

Twum

et al. 2023

Preprint

View full text Add to dashboard Cite

Focus on predictive algorithm and its performance evaluation is extensively covered in most research studies. Best predictive models offer Optimum prediction solutions in the form of prediction accuracy scores, precision, recall etc. Prediction accuracy score from performance evaluation have been used as a determining factor for appropriate model recommendations use. It is one of the most widely used metric for identifying optimal prediction solutions irrespective of context or nature of dataset, size and output class distributions between the minority and majority variables. The key research question however is the impact of using prediction accuracy as compared to balanced accuracy in the determination of model performance in healthcare and other real-world application systems. Answering this question requires an appraisal of current state of knowledge in both prediction accuracy and balanced accuracy use in real-world applications including a search for related works that highlight appropriate machine learning methodologies and techniques.

show abstract

Section: Balanced Accuracy Process Diagrammentioning

confidence: 99%

A systematic review of prediction accuracy as an evaluation measure for determining machine learning model performance in healthcare systems

Owusu-Adjei

Hayfron-Acquah

Twum

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…With respect to clinical usefulness, it could be argued that every new tool should be evaluated within the context of the current standards and its value should be realistically demonstrated in terms of integration rather than substitution [ 64 ]. Establishing this incremental value requires, among others, cost-benefit analyses, holistic models (integrating imaging, clinical and radiomics data), as well as direct comparisons with valid alternatives (e.g., sentinel lymph node biopsy for nodal assessment).…”

Section: Still More Challenges Than Opportunitiesmentioning

confidence: 99%

Artificial Intelligence and Radiomics for Endometrial Cancer MRI: Exploring the Whats, Whys and Hows

Leo,

Stanzione,

Miele

et al. 2023

JCM

Self Cite

View full text Add to dashboard Cite

Endometrial cancer (EC) is intricately linked to obesity and diabetes, which are widespread risk factors. Medical imaging, especially magnetic resonance imaging (MRI), plays a major role in EC assessment, particularly for disease staging. However, the diagnostic performance of MRI exhibits variability in the detection of clinically relevant prognostic factors (e.g., deep myometrial invasion and metastatic lymph nodes assessment). To address these challenges and enhance the value of MRI, radiomics and artificial intelligence (AI) algorithms emerge as promising tools with a potential to impact EC risk assessment, treatment planning, and prognosis prediction. These advanced post-processing techniques allow us to quantitatively analyse medical images, providing novel insights into cancer characteristics beyond conventional qualitative image evaluation. However, despite the growing interest and research efforts, the integration of radiomics and AI to EC management is still far from clinical practice and represents a possible perspective rather than an actual reality. This review focuses on the state of radiomics and AI in EC MRI, emphasizing risk stratification and prognostic factor prediction, aiming to illuminate potential advancements and address existing challenges in the field.

show abstract

“…It is important that the development and evaluation of ML techniques are made transparent and interpretable to allay any doubt about its usability in healthcare systems. Predictive model evaluation especially in healthcare and other real-world application systems with class distribution inequality must take into account the peculiarity of the dataset especially when assessing predictive model performance [ 10 ]. Prediction accuracy score show results obtained from both observed and predicted values.…”

Section: 0 Introductionmentioning

confidence: 99%

Imbalanced class distribution and performance evaluation metrics: A systematic review of prediction accuracy for determining model performance in healthcare systems

Owusu-Adjei,

Ben Hayfron-Acquah,

Frimpong

et al. 2023

PLOS Digit Health

View full text Add to dashboard Cite

Focus on predictive algorithm and its performance evaluation is extensively covered in most research studies to determine best or appropriate predictive model with Optimum prediction solution indicated by prediction accuracy score, precision, recall, f1score etc. Prediction accuracy score from performance evaluation has been used extensively as the main determining metric for performance recommendation. It is one of the most widely used metric for identifying optimal prediction solution irrespective of dataset class distribution context or nature of dataset and output class distribution between the minority and majority variables. The key research question however is the impact of class inequality on prediction accuracy score in such datasets with output class distribution imbalance as compared to balanced accuracy score in the determination of model performance in healthcare and other real-world application systems. Answering this question requires an appraisal of current state of knowledge in both prediction accuracy score and balanced accuracy score use in real-world applications where there is unequal class distribution. Review of related works that highlight the use of imbalanced class distribution datasets with evaluation metrics will assist in contextualizing this systematic review.

show abstract

Must-have Qualities of Clinical Research on Artificial Intelligence and Machine Learning

Cited by 25 publications

References 106 publications

A systematic review of prediction accuracy as an evaluation measure for determining machine learning model performance in healthcare systems

A systematic review of prediction accuracy as an evaluation measure for determining machine learning model performance in healthcare systems

Artificial Intelligence and Radiomics for Endometrial Cancer MRI: Exploring the Whats, Whys and Hows

Imbalanced class distribution and performance evaluation metrics: A systematic review of prediction accuracy for determining model performance in healthcare systems

Contact Info

Product

Resources

About