Development and use of reporting guidelines for assessing the quality of validation studies of health administrative data

Benchimol, Eric I; Dujovny, Manuel; To, Teresa; Griffiths, Anne M.; Rabeneck, Linda; Guttmann, Astrid

doi:10.1016/j.jclinepi.2010.10.006

Cited by 304 publications

(403 citation statements)

References 29 publications

Supporting

Mentioning

380

Contrasting

Unclassified

Order By: Relevance

“…With the risk of sampling bias removed, estimation of incidence and prevalence rates can be made, allowing for description of the total burden of a particular disease or outcome, analysis of risk factors and trends, as well as identification of health inequalities and estimation of health costs. 1,2 Accurate conclusions from such analyses rely on high quality data that truly represent the population experience. Assessment of data quality (completeness and accuracy) is typically undertaken by a validation study, in which data from a sample of records from the population dataset are compared to a highly reliable and accurate source of data ('the gold standard') for the corresponding records.…”

mentioning

confidence: 99%

Are pregnancy outcomes associated with risk factor reporting in routinely collected perinatal data?

Ampt¹,

Ford²,

Taylor³

et al. 2013

NSW Public Health Bull.

View full text Add to dashboard Cite

Aim:To assess reporting characteristics of commonly dichotomised pregnancy outcomes (e.g. preterm/term birth); and to investigate whether behaviours (e.g. smoking), medical conditions (e.g. diabetes) or interventions (e.g. induction) were reported differently by pregnancy outcomes. Methods: Further analysis of a previous validation study was undertaken, in which 1680 perinatal records were compared with data extracted from medical records. Continuous and polytomous variables were dichotomised, and risk factor reporting was assessed within the dichotomised outcome groups. Agreement, kappa, sensitivity and positive predictive value calculations were undertaken. Results: Gestational age, birthweight, Apgar scores, perineal trauma, regional analgesia and baby discharge status (live birth/ stillbirth) were reported with high accuracy and reliability when dichotomised (kappa values 0.95-1.00, sensitivities 94.7-100.0%). Although not statistically significant, there were trends for hypertension, infant resuscitation and instrumental birth to be more accurately reported among births with adverse outcomes. In contrast, smoking ascertainment tended to be poorer among preterm births and when babies were ,2500 g. Conclusion: Dichotomising variables collected as continuous or polytomous variables in birth data results in accurate and well ascertained data items. There is no evidence of systematic differential reporting of risk factors.Population level data are well suited to studies evaluating health care. With the risk of sampling bias removed, estimation of incidence and prevalence rates can be made, allowing for description of the total burden of a particular disease or outcome, analysis of risk factors and trends, as well as identification of health inequalities and estimation of health costs. 1,2 Accurate conclusions from such analyses rely on high quality data that truly represent the population experience. Assessment of data quality (completeness and accuracy) is typically undertaken by a validation study, in which data from a sample of records from the population dataset are compared to a highly reliable and accurate source of data ('the gold standard') for the corresponding records. The accuracy and reliability of individual data items are typically reported. 3,4 The variables in perinatal population data can be continuous (e.g. gestational age), nominal (e.g. mode of delivery) and ordinal (e.g. first, second, third or fourth degree perineal tears), with validation of such variables typically reporting percent agreement and kappa statistics. These types of variables are frequently dichotomised in analyses (e.g. preterm birth, caesarean section, or third-fourth degree tears), 5,6 but little assessment has been undertaken into the accuracy and reliability of their dichotomised form.Differential reporting in population health data occurs when a variable is reported with different accuracy and reliability amongst different strata of another variable. This can introduce systematic bias, leading to under or over...

show abstract

mentioning

confidence: 99%

Are pregnancy outcomes associated with risk factor reporting in routinely collected perinatal data?

Ampt¹,

Ford²,

Taylor³

et al. 2013

NSW Public Health Bull.

View full text Add to dashboard Cite

show abstract

“…Sensitivity and PPV were the most common validity measures, which is also consistent with other reviews of administrative health data quality. 35,36 There are a number of gaps in the current body of Canadian studies validating administrative health data. First, the majority of the studies examined databases from Alberta and Ontario.…”

Section: Discussionmentioning

confidence: 99%

Quality of administrative health databases in Canada: A scoping review

Hinds

Lix

Smith

et al. 2016

Can J Public Health

View full text Add to dashboard Cite

OBJECTIVE: Administrative health databases are increasingly used to conduct population-based health research and surveillance; this has resulted in a corresponding growth in studies about their quality. Our objective was to describe the characteristics of published Canadian studies about administrative health database quality.METHODS: PubMed, Scopus, and Google Advanced were searched, along with websites of relevant organizations. English-language studies that evaluated the quality of one or more Canadian administrative health databases between 2004 and 2014 were selected for inclusion. Extracted information included data quality concepts and measures, year and type of publication, type of database, and geographic origin.SYNTHESIS: More than 3,000 publications were identified from the search. Twelve reports and 144 peer-reviewed papers were included. The majority (53.5%) of peer-review publications used databases from Ontario and Alberta, while 67% of the non-peer-review publications used data from multiple provinces/ territories. Almost all peer-reviewed papers (97.2%) were validation studies. Hospital discharge abstracts and physician billing claims were the most frequently validated databases. Approximately half of the publications (53.0%) validated case definitions and 37.7% focused on a chronic physical health condition. CONCLUSION:Gaps in the Canadian administrative data quality literature include a limited number of studies evaluating data from the Maritimes and across multiple jurisdictions, newer data sources, validating methods for identifying individuals with mental illness, and assessing the completeness and serviceability of the data. Data quality studies can aid researchers to understand the strengths and limitations of the data. Administrative health databases were originally created for health care management and monitoring functions, such as remunerating physicians. Although these databases were not originally designed to be used for research and surveillance, they contain a rich source of information that is now routinely used for these purposes.5,6 Therefore a central question underlying studies that use administrative health databases is "Are the data of good quality for their intended use?" Given that administrative data are often used in epidemiological and public health research, it is important to conduct data quality studies so researchers are kept informed about the strengths and limitations of the data, and can then take steps to produce unbiased results by minimizing selection bias and measurement error. These efforts support good decision making about health and health care use. Data quality is a broad concept that is both relative and multidimensional.7 One comprehensive definition is "the totality of features and characteristics of a data set that bear on its ability to satisfy the needs that result from the intended use of the data". The latter concept refers to the ability to bring together data from different sources; for example, coherence can be achieved by using comm...

show abstract

“…Study reporting and methods conform to recommendations for administrative data validation study reporting. 1 Ethics approval was granted by Western University.…”

Section: Methodsmentioning

confidence: 99%

“…Administrative data are defined as data that are routinely collected for a purpose other than a specific research question. 1 Canada's universal health-care system provides complete administrative data for the population, and is ideally suited for observational studies.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An administrative data algorithm to identify traumatic spinal cord injured patients: a validation study

et al. 2013

View full text Add to dashboard Cite

Objective: To assess the validity of different administrative data sources available for the identification of traumatic spinal cord injured (TSCI) patients. Study design: Retrospective validation study. Setting: Ontario, Canada. Participants: Adult patients seen in tertiary outpatient spinal cord rehabilitation clinics after 1 April 2002. Outcome measures: Sensitivity, specificity, positive and negative predicative values of diagnostic ICD10 codes from Canadian Institutes of Health Discharge Abstracts (CIHI-DAD), Rehabilitation Coding Groups (RCG) from that National Rehabilitation System (NRS), and spinal cord injury fee codes from the Ontario Healthcare Insurance Plan (OHIP). Secondary outcome was the agreement between actual lesion level and RCG/ICD10 coded lesion level. Results: The RCG codes in the NRS have high sensitivity (92%, 95% confidence interval (CI): 87-95%) and specificity (97%, 95% CI: 94-99%) for the identification of true TSCI patients, whereas CIHI-DAD ICD10 codes are highly specific (99%, 95% CI: 95-100) and moderately sensitive (76%, 95% CI: 79-87%). OHIP fee codes had poor sensitivity (64%, 95% CI: 57-71%). Agreement between true lesion level and the NRS and CIHI-DAD coding is good (Kappa of 0. 65-0.88 and 0.56-0.70, respectively). Conclusion: This study demonstrated that the NRS is able to accurately discriminate between patients with and without a TSCI. A large population of incident and prevalent TSCI patients are identifiable using administrative data.

show abstract

Development and use of reporting guidelines for assessing the quality of validation studies of health administrative data

Cited by 304 publications

References 29 publications

Are pregnancy outcomes associated with risk factor reporting in routinely collected perinatal data?

Are pregnancy outcomes associated with risk factor reporting in routinely collected perinatal data?

Quality of administrative health databases in Canada: A scoping review

An administrative data algorithm to identify traumatic spinal cord injured patients: a validation study

Contact Info

Product

Resources

About