The importance of data issues when comparing cystic fibrosis registry outcomes between countries: Are annual review FEV₁ in the UK only collected when subjects are well?

Abstract

Rationale, aims and objectiveCross‐country comparisons of cystic fibrosis (CF) outcomes can potentially identify variation in care but are dependent on data quality. An important assumption is that the UK annual review FEV1 is only collected during periods of clinical stability. If this assumption does not hold, results of FEV1 comparisons may be biased in favour of registries with encounter‐based FEV1. We aimed to test the assumption that CF annual reviews in the UK are only performed during periods of clinical stability.MethodProspective encounter‐based data collected in Sheffield (n = 174) was used to establish whether annual review FEV1 were always collected during periods of clinical stability and to determine the group‐level discrepancy between annual review vs best FEV1. We then went on to quantify the group‐level discrepancy between annual review and best annual FEV1 readings within the UK registry (n = 2995) to determine if the differences observed in Sheffield also apply to the wider UK data.ResultsSheffield results showed a group‐level discrepancy between best and annual review FEV1 of −2.5% (95% CI −3.95% to −1.2%) for annual reviews performed during periods of clinical stability (n = 50). The group‐level discrepancy is larger at −8.0% (95% CI −11.2% to −4.9%) among annual reviews performed during periods of clinical instability (n = 13). Therefore, the magnitude of this group‐level discrepancy is a surrogate for the proportion of clinically stable annual reviews—smaller discrepancy indicates a higher proportion of clinically stable annual reviews and vice versa.The overall group‐level discrepancy in the UK registry (−5.6%, 95% CI −5.9 to −5.4%) was similar to Sheffield (−6.1%, 95% CI −7.1 to −5.1%). Around 20% of the clinician reviewed, annual reviews in Sheffield were performed during periods of clinically instability.ConclusionsAnnual review FEV1 underestimates lung health of adults with CF in the UK and may bias cross‐country comparisons.

Investigating the effects of long-term dornase alfa use on lung function using registry data

“…The unknown timing of the best FEV1 measure relative to the other measures would also present additional challenges for the analysis. According the Registry protocol, annual reviews take place at a time when the patient is stable, and ongoing validation procedures indicate good adherence to this protocol [28]. Furthermore, there is no evidence to suggest that those receiving DNase are more likely to have their annual review during an unstable period compared to those not receiving DNase, meaning that using the FEV 1 measure obtained during the annual assessment should not result in any bias.…”

Understanding FEV1 for the purpose of cystic fibrosis registry comparisons: Does bias in annual review FEV1 affect between‐centre comparison within the UK? An analysis of registry data

“…We previously demonstrated that annual review %FEV 1 underestimates lung health of adults in comparison with %FEV 1 captured during periods of clinical stability, which is an important systematic bias to consider for cross‐country comparisons against registries with encounter‐based FEV 1 . In this manuscript, we extended our previous work and analysed centre‐by‐centre discrepancy between paired annual review and best %FEV 1 to determine if systematic bias in annual review FEV 1 also affects between‐centre comparisons within the United Kingdom.…”

“…There has been a move by the UK CF registry towards comparisons using best FEV 1 data (which may be more reliable than annual review FEV 1 ), and it is encouraging that in the 2017 registry annual data report, 25/28 (89%) of centres now have ≥80% completeness for best FEV 1 data . Nonetheless, “best FEV 1 ” reported to the registry could still underestimate the lung health of people with CF if these data did not reliably select the highest FEV 1 from all FEV 1 readings over a 1‐year period . Identification of best FEV 1 data is difficult if data collectors are required to manually extract these data from routine clinical record, particularly in larger centres.…”

“…The magnitude of group‐level mean discrepancy between paired annual review and best %FEV 1 is a surrogate for the extent by which annual review %FEV 1 underestimates lung health of adults with CF—a larger discrepancy indicates greater underestimation of centre‐level lung health and vice versa . Therefore, centre‐by‐centre discrepancy between paired annual review and best %FEV 1 was calculated, then plotted against centre size.…”

“…However, the “pyramid of investigation” model emphasizes that data must first be sufficiently robust before drawing other conclusions (eg, regarding quality of care) in observational epidemiology . Our previous analysis demonstrated that annual review %FEV 1 underestimates lung health of adults with CF in the United Kingdom in comparison with %FEV 1 captured during periods of clinical stability . It is uncertain whether the bias in annual review %FEV 1 affects between‐centre comparison within the United Kingdom.…”