An Ensemble Learning Strategy for Eligibility Criteria Text Classification for Clinical Trial Recruitment: Algorithm Development and Validation

Abstract

<jats:sec> <jats:title>Background</jats:title> <jats:p>Eligibility criteria are the main strategy for screening appropriate participants for clinical trials. Automatic analysis of clinical trial eligibility criteria by digital screening, leveraging natural language processing techniques, can improve recruitment efficiency and reduce the costs involved in promoting clinical research.</jats:p> </jats:sec> <jats:sec> <jats:title>Objective</jats:title> <jats:p>We aimed to create a natural language processing model to automatically classify clinical trial eligibility criteria.</jats:p> </jats:sec> <jats:sec> <jats:title>Methods</jats:title> <jats:p>We proposed a classifier for short text eligibility criteria based on ensemble learning, where a set of pretrained models was integrated. The pretrained models included state-of-the-art deep learning methods for training and classification, including Bidirectional Encoder Representations from Transformers (BERT), XLNet, and A Robustly Optimized BERT Pretraining Approach (RoBERTa). The classification results by the integrated models were combined as new features for training a Light Gradient Boosting Machine (LightGBM) model for eligibility criteria classification.</jats:p> </jats:sec> <jats:sec> <jats:title>Results</jats:title> <jats:p>Our proposed method obtained an accuracy of 0.846, a precision of 0.803, and a recall of 0.817 on a standard data set from a shared task of an international conference. The macro F1 value was 0.807, outperforming the state-of-the-art baseline methods on the shared task.</jats:p> </jats:sec> <jats:sec> <jats:title>Conclusions</jats:title> <jats:p>We designed a model for screening short text classification criteria for clinical trials based on multimodel ensemble learning. Through experiments, we concluded that performance was improved significantly with a model ensemble compared to a single model. The introduction of focal loss could reduce the impact of class imbalance to achieve better performance.</jats:p> </jats:sec>

Development of a Web-Based Ensemble Machine Learning Application to Select the Optimal Size of Posterior Chamber Phakic Intraocular Lens

“…To solve this problem, ensemble machine learning approaches have been introduced because of their robustness and ability to enhance the performance of machine learning models. 12 Ensemble learning-based approaches have shown very good performance in many recent machine learning competitions. They can reduce the effect of overfitting and outliers by combining the results of several machine learning models.…”

Semantic categorization of Chinese eligibility criteria in clinical trials using machine learning methods

“…To our best knowledge, these works focused on the English eligibility criteria, only an academic conference [ 23 , 24 ] has paid attention to the Chinese eligibility criteria classification. With the exponential accumulation of Chinese electronic medical records [ 25 ] and continued increasing of Chinese clinical trial registration, there is an urgent need to computable characteristic the Chinese eligibility criteria.…”

Health Natural Language Processing: Methodology Development and Applications

“…The classification of clinical trial eligibility criteria texts is a fundamental and critical step in clinical target population recruitment. Zeng et al [ 15 ] proposed an ensemble learning method that integrates the current cutting-edge deep learning models BERT, Enhanced Language Representation with Informative Entities, XLNet, and RoBERT. Through a model ensemble in two layers, the study trained a model and compared it with a list of baseline deep learning models on a publicly available standard data set.…”