PS2-37: Development and Use of a Predictive Analytics Tool in a Large Healthcare Organization

Abstract

Background/Aims: In preliminary work toward Cancer Research Networkbased studies of lymphoma risk pathways, we undertook an evaluation of the utility of ICD-9 CM diagnostic codes to accurately identify potential risk factor conditions, including rheumatoid arthritis (RA). Methods: Using the enrolled Virtual Data Warehouse (VDW) cohort at Marshfield Clinic Research Foundation, we ascertained a set of potential RA cases diagnosed between 2000 and 2010 by the presence of one or more diagnostic codes for RA (ICD-9 CM 714-714.89). Medical records were abstracted by trained staff on a random sample of 206 cases. Cases were adjudicated into categories of confirmed, probable, documented physician diagnosis only, equivocal, and non-case using a literature-based gold standard definition scheme. Outcome measures included positive predictive value (PPV) and relative sensitivity, with the confirmed, probable and physician diagnosis categories considered valid cases for the main analysis. Results: Upon review, 25 subjects did not have sufficient medical records available for evaluation, leaving 181 subjects for analysis, including 57 with only one RA code and 124 with 2 or more instances of an RA code. Overall PPV for patients with one or more RA codes was 56%. Subjects with only one diagnostic code had very low PPV (12%), while PPV was higher among those with 2 or more (76%). PPV improved to 91% when requiring a rheumatologist diagnosis, but only 40% of the true cases in the set were detectable in this way. The one algorithm that provided acceptably high PPV and relative sensitivity required 2 or more RA diagnostic codes and history of a rheumatoid factor test (PPV 83%, relative sensitivity 82%). Conclusions: A single ICD-9CM diagnostic code for RA was not highly predictive of a true diagnosis, but PPV was enhanced when requiring multiple codes and incorporating other VDW data elements. With one exception, algorithms with higher PPV had strong reductions in relative sensitivity. Limitations include non-systematic collection of RA medication data preventing the use of treatment in the algorithms, and the inability to evaluate absolute sensitivity. Next steps include a joint RA analysis with investigators at Kaiser Permanente Southern California, and possible inclusion of RA treatment data into the Marshfield algorithms.