A review of routinely collected data studies in urology: Methodological considerations, reporting quality, and future directions

Abstract

Studies using routinely collected data (RCD) are common in the urological literature; however, there are important considerations in the creation and review of RCD discoveries. A recent reporting guideline (REporting of studies Conducted using Observational Routinely-collected health Data, RECORD) was developed to improve the reporting of these studies. This narrative review examines important considerations for RCD studies. To assess the current level of reporting in the urological literature, we reviewed all the original research articles published in Journal of Urology and European Urology in 2014, and determined the proportion of the RECORD checklist items that were reported for RCD studies. There were 56 RCD studies identified among the 608 articles. When the RECORD items were considered applicable to the specific study, they were reported in 52.5% of cases. Studies most consistently (>80% of them) reported the names of the data sources, the study time frame, the extent to which the authors could access the database source, the patient selection, and discussed missing data. Few studies (<25%) discussed validation of key coding elements, details on data-linkage, data-cleaning, the impact of changing eligibility over time, or provided the complete list of coding elements used to define key study variables. Reporting factors specifically relevant in RCD studies may serve to increase the quality of these studies in the urological literature. With increased technological integration in healthcare and the proliferation of electronic medical records, RCD will continue to be an important source for urological research. IntroductionRoutinely collected data (RCD, also known as administrative data) is increasingly being used to answer clinical research questions. While there are many sources of observational data (for example, population surveys or patient registries), RCD is specifically defined as data that is routinely collected for a purpose other than research. Common examples of RCD sources include physician billing records, insurance company records, or government-mandated healthcare utilization records. 1There are several important strengths that have driven the increased use of RCD research over the last two decades. These include low study costs (especially when compared to prospective clinical studies or randomized, controlled trials), rapid delivery of results, substantial sample sizes, improved generalizability, long-term patient followup (often across different hospitals and physician practices), and the ability to identify very rare patient populations or outcomes.Given the apparent increase in publications using RCD in many urological journals, we sought to review several key issues related to this research area that are relevant for those conducting these studies, and for clinicians who wish to critically read these studies, understand potential methodological limitations, and evaluate the strength of the study's conclusions. Methodological considerations for the use of RCDAny study using RCD ...

Data source profile reporting by studies that use routinely collected health data to explore the effects of drug treatment

“…Similar to our findings, a survey of 124 RCD studies published in 2012 found that only 29.3% of studies adequately reported data linkage [18]. Another study of 56 urological manuscripts published in 2014 showed that 48.2% reported the geographic region of the database, and none reported the methods used to link the data [19]. This study found that data source characteristics differed between those studies published in the top five general medical journals and those published in lower-level medical journals.…”

“…Previous studies have shown that underreporting of data sources is common [16][17][18]. However, many of these studies were either outdated or used a small sample size [17][18][19], and none focused on research exploring the effects of drug treatment. For example, a literature review of 25 studies that used RCD for pharmacovigilance found that only 44% reported the type of data source [17].…”

Validation of acute myocardial infarction (AMI) in electronic medical records: the SPEED-EXTRACT Study

“…In particular, we found a 12 percent false negative rate for STEMI and a 31 percent false negative rate for NSTEMI. The high proportion of missed cases among ICD-10 codes adds weight to calls from other researchers to use routinely collected health care data for research and reporting, rather than rely on administrative data alone (Nissen et al, 2019; Welk and Kwong, 2017; Xu et al, 2020).…”

“…These clinicians are commonly the most junior members of the clinical team, usually in training, and errors or omissions in diagnoses are rarely reviewed or corrected (Nicholls et al, 2017; Tang et al, 2017). Despite this, medical records have traditionally been the clinical reference standard against which ICD-10 codes are validated [Welk and Kwong (2017); Wiegersma et al (2020);]. Furthermore as more than one recent review has pointed out (McCormick et al, 2014; Metcalfe et al, 2012; Rubbo et al, 2015), for practical reasons most validation studies are restricted to cases with an ICD-10 code for AMI.…”

Routinely collected data for population-based outcomes research

“…Ideally these key variables such as the primary outcome should have known measurement characteristics (such as a positive predictive value) so that you can judge how well that code represents what it is meant to represent. This has traditionally been poorly done, [2][3][4] and when it is done this elevates administrative data studies to a higher level.…”