Field testing a new ICD coding system: methods and early experiences with ICD-11 Beta Version 2018

Abstract

Objective A beta version (2018) of International Classification of Diseases, 11th Revision for MMS (ICD-11), needed testing. Field-testing involves real-world application of the new codes to examine usability. We describe creating a dataset and characterizing the usability of ICD-11 code set by coders. We compare ICD-11 against ICD-10-CA (Canadian modification) and a reference standard dataset of diagnoses. Real-world usability encompasses code selection and time to code a complete inpatient chart using ICD-11 compared with ICD-10-CA. Methods and results A random sample of inpatient records previously coded using ICD-10-CA was selected from hospitals in Calgary, Alberta (N = 2896). Nurses examined these charts for conditions and healthcare-related harms. Clinical coders re-coded the same charts using ICD-11 codes. Inter-rater reliability (IRR) and coding time improved with ICD-11 coding experience (23.6 to 9.9 min average per chart). Code structure comparisons and challenges encountered are described. Overall, 86.3% of main condition codes matched. Coder comments regarding duplicate codes, missing codes, code finding issues enabled improvements to the ICD-11 Browser, Coding Tool, and Reference Guide. Training is essential for solid IRR with 17,000 diagnostic categories in the new ICD-11. As countries transition to ICD-11, our coding experiences and methods can inform users for implementation or field testing.

Hypertension identification using inpatient clinical notes from electronic medical records: an explainable, data-driven algorithm study

“…We conducted a retrospective cohort study in which we reviewed medical charts to determine a reference standard for hypertension status in a cohort of randomly selected patients who were admitted to 3 acute care centres in Calgary, Alberta. 13 We compared the performance of EMR phenotype algorithms developed using machine learning and an ICD-based algorithm in identifying patients with hypertension. We used machine learning to develop an EMR phenotype for hypertension and create a data-driven, rule-based algorithm.…”

“…We used data from a chart review cohort of patients who were selected for a field trial of the new International Statistical Classification of Diseases and Related Health Problems , 11th Revision for Mortality and Morbidity Statistics (ICD-11) diagnosis coding system. 13 The patients in this cohort were at least 18 years of age and were discharged from 1 of 3 acute care facilities in Calgary, between Jan. 1 and June 30, 2015. The 3 facilities involved were an academic hospital (Foothills Medical Centre, affiliated with the University of Calgary) and 2 community hospitals (Peter Lougheed Centre and Rockyview General Hospital).…”

“…For patients with several admissions, 1 admission within the study period was randomly selected. A sample-size calculation from the ICD-11 field trial study 13 determined that 3000 records were required to detect a 10% difference in sensitivity between ICD-10-CA and ICD-11 coding of common comorbidities, such as hypertension, using Lachenbruch’s midpoint method 15 based on prevalence results from previous findings. 3 …”

“…This provided the reference standard for hypertension that both our algorithm and the existing ICD algorithm were compared against. We extracted patient charts for each of the included admissions from the hospital records departments, 13 which provided EMR data from SCM as well as paper charts not necessarily a part of the EMR. Trained chart reviewers (all nurses) looked for any form of listed diagnosis of hypertension (e.g., controlled hypertensives or essential hypertension) in patients’ history and physical, multidisciplinary progress notes, consult notes and discharge summary.…”

“…The inter-rater reliability between reviewers was high (> 0.8 κ). 13 These hypertension labels were used as a reference standard for both our machine-learning algorithms and for the ICD-based algorithm.…”