No abstract

ImportanceThe absence of reliable hospital discharge data regarding the intent of firearm injuries (ie, whether caused by assault, accident, self-harm, legal intervention, or an act of unknown intent) has been characterized as a glaring gap in the US firearms data infrastructure.ObjectiveTo use incident-level information to assess the accuracy of intent coding in hospital data used for firearm injury surveillance.Design, Setting, and ParticipantsThis cross-sectional retrospective medical review study was conducted using case-level data from 3 level I US trauma centers (for 2008-2019) for patients presenting to the emergency department with an incident firearm injury of any severity.ExposuresClassification of firearm injury intent.Main Outcomes and MeasuresResearchers reviewed electronic health records for all firearm injuries and compared intent adjudicated by team members (the gold standard) with International Classification of Diseases, Ninth and Tenth Revision, Clinical Modification (ICD-9-CM and ICD-10-CM) codes for firearm injury intent assigned by medical records coders (in discharge data) and by trauma registrars. Accuracy was assessed using intent-specific sensitivity and positive predictive value (PPV).ResultsOf the 1227 cases of firearm injury incidents seen during the ICD-10-CM study period (October 1, 2015, to December 31, 2019), the majority of patients (1090 [88.8%]) were male and 547 (44.6%) were White. The research team adjudicated 837 (68.2%) to be assaults. Of these assault incidents, 234 (28.0%) were ICD coded as unintentional injuries in hospital discharge data. These miscoded patient cases largely accounted for why discharge data had low sensitivity for assaults (66.3%) and low PPV for unintentional injuries (34.3%). Misclassification was substantial even for patient cases described explicitly as assaults in clinical notes (sensitivity of 74.3%), as well as in the ICD-9-CM study period (sensitivity of 77.0% for assaults and PPV of 38.0% for unintentional firearm injuries). By contrast, intent coded by trauma registrars differed minimally from researcher-adjudicated intent (eg, sensitivity for assault of 96.0% and PPV for unintentional firearm injury of 93.0%).Conclusions and RelevanceThe findings of this cross-sectional study underscore questions raised by prior work using aggregate count data regarding the accuracy of ICD-coded discharge data as a source of firearm injury intent. Based on our observations, researchers and policy makers should be aware that databases drawn from hospital discharge data (most notably, the Nationwide Emergency Department Sample) cannot be used to reliably count or characterize intent-specific firearm injuries.

ImportanceInternational Classification of Diseases–coded hospital discharge data do not accurately reflect whether firearm injuries were caused by assault, unintentional injury, self-harm, legal intervention, or were of undetermined intent. Applying natural language processing (NLP) and machine learning (ML) techniques to electronic health record (EHR) narrative text could be associated with improved accuracy of firearm injury intent data.ObjectiveTo assess the accuracy with which an ML model identified firearm injury intent.Design, Setting, and ParticipantsA cross-sectional retrospective EHR review was conducted at 3 level I trauma centers, 2 from health care institutions in Boston, Massachusetts, and 1 from Seattle, Washington, between January 1, 2000, and December 31, 2019; data analysis was performed from January 18, 2021, to August 22, 2022. A total of 1915 incident cases of firearm injury in patients presenting to emergency departments at the model development institution and 769 from the external validation institution with a firearm injury code assigned according to International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) or International Statistical Classification of Diseases and Related Health Problems, 10th Revision, Clinical Modification (ICD-10-CM), in discharge data were included.ExposuresClassification of firearm injury intent.Main Outcomes and MeasuresIntent classification accuracy by the NLP model was compared with ICD codes assigned by medical record coders in discharge data. The NLP model extracted intent-relevant features from narrative text that were then used by a gradient-boosting classifier to determine the intent of each firearm injury. Classification accuracy was evaluated against intent assigned by the research team. The model was further validated using an external data set.ResultsThe NLP model was evaluated in 381 patients presenting with firearm injury at the model development site (mean [SD] age, 39.2 [13.0] years; 348 [91.3%] men) and 304 patients at the external development site (mean [SD] age, 31.8 [14.8] years; 263 [86.5%] men). The model proved more accurate than medical record coders in assigning intent to firearm injuries at the model development site (accident F-score, 0.78 vs 0.40; assault F-score, 0.90 vs 0.78). The model maintained this improvement on an external validation set from a second institution (accident F-score, 0.64 vs 0.58; assault F-score, 0.88 vs 0.81). While the model showed some degradation between institutions, retraining the model using data from the second institution further improved performance on that site’s records (accident F-score, 0.75; assault F-score, 0.92).Conclusions and RelevanceThe findings of this study suggest that NLP ML can be used to improve the accuracy of firearm injury intent classification compared with ICD-coded discharge data, particularly for cases of accident and assault intents (the most prevalent and commonly misclassified intent types). Future research could refine this model using larger and more diverse data sets.