Automated clinical trial eligibility prescreening: increasing the efficiency of patient identification for clinical trials in the emergency department

Abstract

Objectives (1) To develop an automated eligibility screening (ES) approach for clinical trials in an urban tertiary care pediatric emergency department (ED); (2) to assess the effectiveness of natural language processing (NLP), information extraction (IE), and machine learning (ML) techniques on real-world clinical data and trials.Data and methods We collected eligibility criteria for 13 randomly selected, disease-specific clinical trials actively enrolling patients between January 1, 2010 and August 31, 2012. In parallel, we retrospectively selected data fields including demographics, laboratory data, and clinical notes from the electronic health record (EHR) to represent profiles of all 202795 patients visiting the ED during the same period. Leveraging NLP, IE, and ML technologies, the automated ES algorithms identified patients whose profiles matched the trial criteria to reduce the pool of candidates for staff screening. The performance was validated on both a physician-generated gold standard of trial–patient matches and a reference standard of historical trial–patient enrollment decisions, where workload, mean average precision (MAP), and recall were assessed.Results Compared with the case without automation, the workload with automated ES was reduced by 92% on the gold standard set, with a MAP of 62.9%. The automated ES achieved a 450% increase in trial screening efficiency. The findings on the gold standard set were confirmed by large-scale evaluation on the reference set of trial–patient matches.Discussion and conclusion By exploiting the text of trial criteria and the content of EHRs, we demonstrated that NLP-, IE-, and ML-based automated ES could successfully identify patients for clinical trials.

Cohort Selection for Clinical Trials From Longitudinal Patient Records: Text Mining Approach

“…The development of our system incurred less than 2 person-months, while achieving performance that could boost the recruitment. The system is expected to reduce clinicians’ workload in line with the estimates reported by other studies [11,16,42,43].…”

Cohort Selection for Clinical Trials From Longitudinal Patient Records: Text Mining Approach

“…The development of our system incurred less than 2 person-months, while achieving performance that could boost the recruitment. The system is expected to reduce clinicians’ workload in line with the estimates reported by other studies [11,16,42,43].…”

Cohort Selection for Clinical Trials From Longitudinal Patient Records: Text Mining Approach (Preprint)

“…The system is expected to reduce clinicians' workload in line with the estimates reported by other studies [11,16,42,43].…”

Development of a conceptual framework for defining trial efficiency

“…Only 8(10%) studies had explicitly defined or explained what ‘efficiency’ meant in the context of their trials (see S2 Table for details). We categorised discussions of efficiency from the literature into nine key items: 1)scientific efficiency [ 11 , 19 , 20 ], 2)operational efficiency [ 11 , 20 , 21 ], 3)statistical efficiency [ 14 , 22 – 24 ] and 4)economic efficiency [ 15 , 25 ], 5)efficiency in trial designs [ 7 , 8 , 23 , 26 – 45 ], 6)trial conduct [ 11 , 20 , 21 , 46 – 66 ], and other aspects such as 7)improving efficiency using information technologies and mobile apps [ 53 , 67 – 70 ]; 8)involving the public and stakeholders [ 20 , 71 ]; and 9)efficient trial reviews and regulatory approvals [ 28 , 66 , 72 – 74 ]. (see Table 1 for details).…”

Cohort Selection for Clinical Trials From Longitudinal Patient Records: Text Mining Approach

“…The development of our system incurred less than 2 person-months, while achieving performance that could boost the recruitment. The system is expected to reduce clinicians’ workload in line with the estimates reported by other studies [11,16,42,43].…”

Cohort Selection for Clinical Trials From Longitudinal Patient Records: Text Mining Approach (Preprint)

“…The system is expected to reduce clinicians' workload in line with the estimates reported by other studies [11,16,42,43].…”

Development of a conceptual framework for defining trial efficiency

“…Only 8(10%) studies had explicitly defined or explained what ‘efficiency’ meant in the context of their trials (see S2 Table for details). We categorised discussions of efficiency from the literature into nine key items: 1)scientific efficiency [ 11 , 19 , 20 ], 2)operational efficiency [ 11 , 20 , 21 ], 3)statistical efficiency [ 14 , 22 – 24 ] and 4)economic efficiency [ 15 , 25 ], 5)efficiency in trial designs [ 7 , 8 , 23 , 26 – 45 ], 6)trial conduct [ 11 , 20 , 21 , 46 – 66 ], and other aspects such as 7)improving efficiency using information technologies and mobile apps [ 53 , 67 – 70 ]; 8)involving the public and stakeholders [ 20 , 71 ]; and 9)efficient trial reviews and regulatory approvals [ 28 , 66 , 72 – 74 ]. (see Table 1 for details).…”

Cohort Selection for Clinical Trials From Longitudinal Patient Records: Text Mining Approach

“…The development of our system incurred less than 2 person-months, while achieving performance that could boost the recruitment. The system is expected to reduce clinicians’ workload in line with the estimates reported by other studies [11,16,42,43].…”

Cohort Selection for Clinical Trials From Longitudinal Patient Records: Text Mining Approach (Preprint)

“…The system is expected to reduce clinicians' workload in line with the estimates reported by other studies [11,16,42,43].…”

Development of a conceptual framework for defining trial efficiency

“…Only 8(10%) studies had explicitly defined or explained what ‘efficiency’ meant in the context of their trials (see S2 Table for details). We categorised discussions of efficiency from the literature into nine key items: 1)scientific efficiency [ 11 , 19 , 20 ], 2)operational efficiency [ 11 , 20 , 21 ], 3)statistical efficiency [ 14 , 22 – 24 ] and 4)economic efficiency [ 15 , 25 ], 5)efficiency in trial designs [ 7 , 8 , 23 , 26 – 45 ], 6)trial conduct [ 11 , 20 , 21 , 46 – 66 ], and other aspects such as 7)improving efficiency using information technologies and mobile apps [ 53 , 67 – 70 ]; 8)involving the public and stakeholders [ 20 , 71 ]; and 9)efficient trial reviews and regulatory approvals [ 28 , 66 , 72 – 74 ]. (see Table 1 for details).…”