Internal and external validation of an updated ICD-10-CA to AIS-2005 update 2008 algorithm

Abstract

BACKGROUND Administrative data are a powerful tool for population-level trauma research but lack the trauma-specific diagnostic and injury severity codes needed for risk-adjusted comparative analyses. The objective of this study was to validate an algorithm to derive Abbreviated Injury Scale (AIS-2005 update 2008) severity scores from Canadian International Classification of Diseases (ICD-10-CA) diagnostic codes in administrative data. METHODS This was a retrospective cohort study using data from the 2009 to 2017 Ontario Trauma Registry for the internal validation of the algorithm. This registry includes all patients treated at a trauma center who sustained a moderate or severe injury or were assessed by a trauma team. It contains both ICD-10-CA codes and injury scores assigned by expert abstractors. We used Cohen's kappa (𝜅) coefficient to compare AIS-2005 Update 2008 scores assigned by expert abstractors to those derived using the algorithm and the intraclass correlation coefficient to compare assigned and derived Injury Severity Scores. Sensitivity and specificity for detection of a severe injury (AIS score, ≥ 3) were then calculated. For the external validation of the algorithm, we used administration data to identify adults who either died in an emergency department or were admitted to hospital in Ontario secondary to a traumatic injury (2009–2017). Logistic regression was used to evaluate the discriminative ability and calibration of the algorithm. RESULTS Of 41,869 patients in the Ontario Trauma Registry, 41,793 (99.8%) had at least one diagnosis matched to the algorithm. Evaluation of AIS scores assigned by expert abstractors and those derived using the algorithm demonstrated a high degree of agreement in identification of patients with at least one severe injury (𝜅 = 0.75; 95% confidence interval [CI], 0.74–0.76). Likewise, algorithm-derived scores had a strong ability to rule in or out injury with AIS ≥ 3 (specificity, 78.5%; 95% CI, 77.7–79.4; sensitivity, 95.1; 95% CI, 94.8–95.3). There was strong correlation between expert abstractor-assigned and crosswalk-derived Injury Severity Score (intraclass correlation coefficient, 0.80; 95% CI, 0.80–0.81). Among the 130,542 patients identified using administrative data, the algorithm retained its discriminative properties. CONCLUSION Our ICD-10-CA to AIS-2005 update 2008 algorithm produces reliable estimates of injury severity and retains its discriminative properties with administrative data. Our findings suggest that this algorithm can be used for risk adjustment of injury outcomes when using population-based administrative data. LEVEL OF EVIDENCE Diagnostic Tests/Criteria; Level II.