Encounter Appropriateness Score for You Model: Development and Pilot Implementation of a Predictive Model to Identify Visits Appropriate for Telehealth in Rheumatology

Abstract

ObjectiveDevelop a decision‐making tool to predict telehealth appropriateness for future Rheumatology visits and expand telehealth care access.MethodsThe model was developed using the Encounter Appropriateness Score for You (EASY) and Electronic Health Record (EHR) data at a single academic rheumatology practice from January 1st, 2021, to December 31st, 2021. The EASY model is a logistic regression model that includes encounter characteristics, patient sociodemographic and clinical characteristics, and provider characteristics. The goal of pilot implementation was to determine if model recommendations align with provider preferences and influence telehealth scheduling. Four providers were presented with future encounters that the model identified as candidates for a change in encounter modality (true changes), along with an equal number of artificial (false) recommendations. Providers and patients could accept or reject proposed changes.ResultsThe model performs well with an AUC from 0.831 – 0.855 in 21,679 encounters across multiple validation sets. Covariates that contributed most to model performance were provider preference for and frequency of telehealth encounters. Other significant contributors included encounter characteristics (current scheduled encounter modality) and patient factors (age, RAPID3 scores, diagnoses, medications). The pilot included 201 encounters. Providers were more likely to agree with true versus artificial recommendations (Cohen's Kappa = 0.45, p < 0.001), and the model increased the number of appropriate telehealth visits.ConclusionThe EASY model accurately identifies future visits that are appropriate for telehealth. This tool can support shared decision‐making between patients and providers in deciding the most appropriate follow‐up encounter modality.

Evaluating Provider Perceptions of Telehealth Utility in Outpatient Rheumatology Telehealth Encounters

“…We recently used the insights gained from the TUS and EASY scoring systems to create a predictive model that accurately triages patients with rheumatic diseases to receive telehealth versus in-person care in the outpatient setting. 27 In the future, we also plan to investigate patient perceptions of telehealth utility and how these perceptions compare to those of their providers.…”