Using routinely recorded data in a UK RCT: a comparison to standard prospective data collection methods

Abstract

Background Routinely recorded data held in electronic health records can be used to inform the conduct of randomised controlled trials (RCTs). However, limitations with access and accuracy have been identified. Objective: Using epilepsy as an exemplar condition, we assessed the attributes and agreement of routinely recorded data compared to data collected using case report forms in a UK RCT assessing antiepileptic drug treatments for individuals newly diagnosed with epilepsy. Methods The case study RCT is the Standard and New Antiepileptic Drugs II (SANAD II) trial, a pragmatic, UK multicentre RCT assessing the clinical and cost-effectiveness of antiepileptic drugs as treatments for epilepsy. Ninety-eight of 470 eligible participants provided consent for access to routinely recorded secondary care data that were retrieved from NHS Digital Hospital Episode Statistics (N=71) and primary and secondary care data from The Secure Anonymised Information Linkage Databank (N=27). We assessed data items relevant to the identification of individuals eligible for inclusion in SANAD II, baseline and follow-up visits. The attributes of routinely recorded data were assessed including the degree of missing data. The agreement between routinely recorded data and data collected on case report forms in SANAD II was assessed using calculation of Cohen’s kappa for categorical data and construction of Bland-Altman plots for continuous data. Results There was a significant degree of missing data in the routine record for 15 of the 20 variables assessed, including all clinical variables. Agreement was poor for the majority of comparisons, including the assessments of seizure occurrence and adverse events. For example, only 23/62 (37%) participants had a date of first-ever seizure identified in routine datasets. Agreement was satisfactory for the date of prescription of antiepileptic drugs and episodes of healthcare resource use. Conclusions There are currently significant limitations preventing the use of routinely recorded data for participant identification and assessment of clinical outcomes in epilepsy, and potentially other chronic conditions. Further research is urgently required to assess the attributes, agreement, additional benefits, cost-effectiveness and ‘optimal mix’ of routinely recorded data compared to data collected using standard methods such as case report forms at clinic visits for people with epilepsy. Trial registration Standard and New Antiepileptic Drugs II (SANAD II (EudraCT No: 2012-001884-64, registered 05/09/2012; ISRCTN Number: ISRCTN30294119, registered 03/07/2012))

Accuracy of efficient data methods to determine the incidence of hospital-acquired thrombosis and major bleeding in medical and surgical inpatients: a multicentre observational cohort study in four UK hospitals

“… 27 28 Recent case studies have reported significant amounts of missing data and poor interobserver agreement between routinely collected EHR data accessible through HES and case report form evaluation. 29 Electronic records contain an abundance of free text, but often lack necessary intelligence to classify patient episodes appropriately, or allow processing and comparison of routinely collected data. 30 Increasing complexity in outcome is also likely to correspond with decreasing accuracy of routine data.…”

“…For example, relevant data may contain coding errors arising from ambiguous documentation by physicians and inconsistent definitions 27 28. Recent case studies have reported significant amounts of missing data and poor interobserver agreement between routinely collected EHR data accessible through HES and case report form evaluation 29. Electronic records contain an abundance of free text, but often lack necessary intelligence to classify patient episodes appropriately, or allow processing and comparison of routinely collected data 30.…”

Augmenting clinical trial economic analysis by linking cancer trial data to administrative data: current landscape and future opportunities

“… 9 However, concordance for some clinical data points (such as comorbidities) depends on the quality of the administrative data set. 11 …”

Evolving a national clinical trials learning health system

“…It is distinct from other approaches in integrating trials into an LHS, such as embedded trials or pragmatic trials, in both its objectives and scope. 19 , 20 , 21 , 22 It expands on prior calls for results data sharing by emphasizing the need to continuously analyze and improve trials while they are ongoing, as opposed to lagging years behind the completion of trials. 14 A Trials LHS represents a vision of integrating data, analysis, and action to improve clinical trial design and conduct using the conceptual model of the learning health cycle.…”

“…Developing a national Trials LHS could help overcome these barriers and improve clinical trials. It is distinct from other approaches in integrating trials into an LHS, such as embedded trials or pragmatic trials, in both its objectives and scope 19‐22 . It expands on prior calls for results data sharing by emphasizing the need to continuously analyze and improve trials while they are ongoing, as opposed to lagging years behind the completion of trials 14 .…”