Assessing and mitigating the effects of class imbalance in machine learning with application to X-ray imaging

Qu, Wendi; Balki, Indranil; Mendez, Mauro; Valen, John; Levman, Jacob; Tyrrell, Pascal N.

doi:10.1007/s11548-020-02260-6

Cited by 34 publications

(21 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Together with dataset size, class imbalance potentially impacts results [ 27 ]. The impact of sample size and class imbalance is a recognized problem in machine learning in radiology but has not been fully explored [ 28 , 29 ]. For NLP in medical texts, the impact of prevalence on model performance is also recognized [ 30 ].…”

Section: Introductionmentioning

confidence: 99%

Deep Learning-Based Natural Language Processing in Radiology: The Impact of Report Complexity, Disease Prevalence, Dataset Size, and Algorithm Type on Model Performance

2021

View full text Add to dashboard Cite

In radiology, natural language processing (NLP) allows the extraction of valuable information from radiology reports. It can be used for various downstream tasks such as quality improvement, epidemiological research, and monitoring guideline adherence. Class imbalance, variation in dataset size, variation in report complexity, and algorithm type all influence NLP performance but have not yet been systematically and interrelatedly evaluated. In this study, we investigate these factors on the performance of four types [a fully connected neural network (Dense), a long short-term memory recurrent neural network (LSTM), a convolutional neural network (CNN), and a Bidirectional Encoder Representations from Transformers (BERT)] of deep learning-based NLP. Two datasets consisting of radiologist-annotated reports of both trauma radiographs (n = 2469) and chest radiographs and computer tomography (CT) studies (n = 2255) were split into training sets (80%) and testing sets (20%). The training data was used as a source to train all four model types in 84 experiments (Fracture-data) and 45 experiments (Chest-data) with variation in size and prevalence. The performance was evaluated on sensitivity, specificity, positive predictive value, negative predictive value, area under the curve, and F score. After the NLP of radiology reports, all four model-architectures demonstrated high performance with metrics up to > 0.90. CNN, LSTM, and Dense were outperformed by the BERT algorithm because of its stable results despite variation in training size and prevalence. Awareness of variation in prevalence is warranted because it impacts sensitivity and specificity in opposite directions.

show abstract

Section: Introductionmentioning

confidence: 99%

Deep Learning-Based Natural Language Processing in Radiology: The Impact of Report Complexity, Disease Prevalence, Dataset Size, and Algorithm Type on Model Performance

2021

View full text Add to dashboard Cite

show abstract

“…Carrying out classification at the present state can make the classifier's decision biased towards the majority class. Qu et al assessed the effects of class imbalance and how to mitigate them using two chest X-ray datasets [21]. We augmented the samples of the two minor classes (classes H and V) by 180°and added the resultant images to the original ones to mitigate this issue.…”

Section: Methodsmentioning

confidence: 99%

A Pneumonia Diagnosis Scheme Based on Hybrid Features Extracted from Chest Radiographs Using an Ensemble Learning Algorithm

Masud

Bairagi

Sikder

et al. 2021

Journal of Healthcare Engineering

View full text Add to dashboard Cite

Pneumonia is a fatal disease responsible for almost one in five child deaths worldwide. Many developing countries have high mortality rates due to pneumonia because of the unavailability of proper and timely diagnostic measures. Using machine learning-based diagnosis methods can help to detect the disease early and in less time and cost. In this study, we proposed a novel method to determine the presence of pneumonia and identify its type (bacterial or viral) through analyzing chest radiographs. We performed a three-class classification based on features containing diverse information of the samples. After using an augmentation technique to balance the dataset’s sample sizes, we extracted the chest X-ray images’ statistical features, as well as global features by employing a deep learning architecture. We then combined both sets of features and performed the final classification using the RandomForest classifier. A feature selection method was also incorporated to identify the features with the highest relevance. We tested the proposed method on a widely used (but relabeled) chest radiograph dataset to evaluate its performance. The proposed model can classify the dataset’s samples with an 86.30% classification accuracy and 86.03% F-score, which assert the model’s efficacy and reliability. However, results show that the classifier struggles while distinguishing between viral and bacterial pneumonia samples. Implementing this method will provide a fast and automatic way to detect pneumonia in a patient and identify its type.

show abstract

“…While there are methods to combat this, the best strategy is to increase the amount of data available for training whenever possible [15][16][17][18] .…”

Section: Shallow and Deep Learningmentioning

confidence: 99%

Understanding Artificial Intelligence and Predictive Analytics

et al. 2022

View full text Add to dashboard Cite

Machine learning and artificial intelligence have seen tremendous growth in recent years and have been applied in numerous studies in the field of orthopaedics.Machine learning will soon become critical in the day-to-day operations of orthopaedic practice; therefore, it is imperative that providers become accustomed to and familiar with not only the terminology but also the fundamental techniques behind the technology.A foundation of knowledge regarding machine learning is critical for physicians so they can begin to understand the details in the algorithms that are being developed, which provide improved accuracy compared with clinicians, decreased time required, and a heightened ability to triage patients.

show abstract

Assessing and mitigating the effects of class imbalance in machine learning with application to X-ray imaging

Cited by 34 publications

References 13 publications

Deep Learning-Based Natural Language Processing in Radiology: The Impact of Report Complexity, Disease Prevalence, Dataset Size, and Algorithm Type on Model Performance

Deep Learning-Based Natural Language Processing in Radiology: The Impact of Report Complexity, Disease Prevalence, Dataset Size, and Algorithm Type on Model Performance

A Pneumonia Diagnosis Scheme Based on Hybrid Features Extracted from Chest Radiographs Using an Ensemble Learning Algorithm

Understanding Artificial Intelligence and Predictive Analytics

Contact Info

Product

Resources

About