Identification of Axial Spondyloarthritis Patients in a Large Dataset: The Development and Validation of Novel Methods

Abstract

Objective.Observational axial spondyloarthritis (axSpA) research in large datasets has been limited by a lack of adequate methods for identifying patients with axSpA, because there are no billing codes in the United States for most subtypes of axSpA. The objective of this study was to develop methods to accurately identify patients with axSpA in a large dataset.Methods.The study population included 600 chart-reviewed veterans, with and without axSpA, in the Veterans Health Administration between January 1, 2005, and June 30, 2015. AxSpA identification algorithms were developed with variables anticipated by clinical experts to be predictive of an axSpA diagnosis [demographics, billing codes, healthcare use, medications, laboratory results, and natural language processing (NLP) for key SpA features]. Random Forest and 5-fold cross validation were used for algorithm development and testing in the training subset (n = 451). The algorithms were additionally tested in an independent testing subset (n = 149).Results.Three algorithms were developed: Full algorithm, High Feasibility algorithm, and Spond NLP algorithm. In the testing subset, the areas under the curve with the receiver-operating characteristic analysis were 0.96, 0.94, and 0.86, for the Full algorithm, High Feasibility algorithm, and Spond NLP algorithm, respectively. Algorithm sensitivities ranged from 85.0% to 95.0%, specificities from 78.0% to 93.6%, and accuracies from 82.6% to 91.3%.Conclusion.Novel axSpA identification algorithms performed well in classifying patients with axSpA. These algorithms offer a range of performance and feasibility attributes that may be appropriate for a broad array of axSpA studies. Additional research is required to validate the algorithms in other cohorts.

Natural language processing in low back pain and spine diseases: A systematic review

“…Zhao et al [ 22 ] and Walsh et al [ 23 ] dealt with this this task. The last team also exploited their previous work in their [ 24 ] to identify axSpA patients.…”

Identifying Patients With Axial Spondyloarthritis in Large Datasets: Expanding Possibilities for Observational Research

“…We developed novel methods for identifying axSpA patients in national VHA datasets. Previous validation work with axSpA identification algorithms demonstrated excellent performance in axSpA-enriched populations 16,17 . This study is important for understanding how the algorithms are estimated to perform in the general Veteran population with future studies.…”

“…We also assessed 2 traditional methods that have been used in axSpA epidemiologic studies including ≥2 AS ICD codes ≥7 days apart from any source (AS codes, any specialty) and ≥2 AS ICD codes ≥7 days apart from a rheumatology encounter (AS codes, rheumatology). Details about algorithm development were previously published 16,18 . In brief, the Full Algorithm is the most comprehensive with 3 natural language processing (NLP) models 19 20,21,22,23 .…”

“…Veterans were selected from each risk quartile for chart review. Rheumatologist chart reviewers used annotation software (eHOST 24 ) to review and classify the 300 sampled Veterans as having or not having axSpA, according to expert opinion and our previously published axSpA chart review guidelines 16,18 .…”

“…To address limitations with axSpA patient identification in large datasets, we developed algorithms that use structured and unstructured data to identify axSpA patients in national Veteran Health Administration datasets. These algorithms performed well in a sample of Veterans enriched with risk factors for axSpA (AUC 0.86-0.96) 16 . A similar approach was applied to a Boston-based population enriched with axSpA patients, and the results were similar (AUC 0.80-0.93) 17 .…”