Association of Peer Comparison Emails With Electronic Health Record Documentation of Cancer Stage by Oncologists

PURPOSE The Medicare Access and CHIP Reauthorization Act of 2015 (MACRA) requires eligible clinicians to report clinical quality measures (CQMs) in the Merit-Based Incentive Payment System (MIPS) to maximize reimbursement. To determine whether structured data in electronic health records (EHRs) were adequate to report MIPS CQMs, EHR data aggregated by ASCO's CancerLinQ platform were analyzed. MATERIALS AND METHODS Using the CancerLinQ health technology platform, 19 Oncology MIPS (oMIPS) CQMs were evaluated to determine the presence of data elements (DEs) necessary to satisfy each CQM and the DE percent population with patient data (fill rates). At the time of this analysis, the CancerLinQ network comprised 63 active practices, representing eight different EHR vendors and containing records for more than 1.63 million unique patients with one or more malignant neoplasms (1.73 million cancer cases). RESULTS Fill rates for the 63 oMIPS-associated DEs varied widely among the practices. The average site had at least one filled DE for 52% of the DEs. Only 35% of the DEs were populated for at least one patient record in 95% of the practices. However, the average DE fill rate of all practices was 23%. No data were found at any practice for 22% of the DEs. Since any oMIPS CQM with an unpopulated DE component resulted in an inability to compute the measure, only two (10.5%) of the 19 oMIPS CQMs were computable for more than 1% of the patients. CONCLUSION Although EHR systems had relatively high DE fill rates for some DEs, underfilling and inconsistency of DEs in EHRs render automated oncology MIPS CQM calculations impractical.

show abstract

“…Recording the stage group in EHRs for all cancers is widely considered a best practice, [19][20][21][22]…”

Section: De Analysis Case Study: Stage Groupsmentioning

confidence: 99%

Chasm Between Cancer Quality Measures and Electronic Health Record Data Quality

Schorer¹,

Moldwin

Koskimaki

et al. 2022

JCO Clinical Cancer Informatics

View full text Add to dashboard Cite

show abstract

“…11 We chose to frame information by delivering social comparison feedback, given its low cost and proven success in other clinical contexts. [33][34][35] Although we met our primary outcome, we suspect a greater effect size would be seen by guiding choice through default options (automatic hospice informational visit at a defined time point, such as 6 months estimated by the surprise question 42 or more sophisticated models), 43 enhancing choice (allowing concurrent care), and use of an ongoing (multiple peer comparison reports) versus a single intervention. Much can be learned from concurrent care implementation in Veterans Affairs (VA) medical centers where 68% of patients with cancer use hospice, with 25% using concurrent care.…”

Section: Discussionmentioning

confidence: 99%

“…18 Second, oncologists were compared with peers practicing in the same cancer center, treating the same patient population, and facing similar system-wide barriers. 33 Third, the focus of the intervention and the coupled social norm were a societal endorsed standard. 1 Work from other industries reveals that the combination of peer comparisons and normative appeals can produce longer-run behavior change.…”

Section: Discussionmentioning

confidence: 99%

“…[26][27][28][29][30] Behavioral science research leverages social and cognitive methods to gently nudge clinician decision making while preserving freedom of choice. 31 In cancer care, peer comparisons have effectively improved clinical trial accrual, 32 documentation of cancer staging, 33 prescribing of bonemodifying agents, 34 and serious illness conversations. 35 In this study, a few design strategies may have been particularly important.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Oncologist Peer Comparisons as a Behavioral Science Strategy to Improve Hospice Utilization

Sedhom

Blackford

Gupta

et al. 2022

JCO Oncology Practice

View full text Add to dashboard Cite

PURPOSE: Hospice utilization metrics are essential for any serious effort to improve end-of-life care in oncology. However, oncologists do not routinely receive these personalized reports. We evaluated whether a behavioral science intervention, using peer comparisons coupled with social norms, was associated with improvements in hospice use. METHODS: Oncologists at two academic practices of Johns Hopkins Medicine were randomly assigned to receive a peer comparison report by e-mail displaying individual hospice utilization metrics compared with top-performing peers or to receive no report. The data accrued for the intervention represented hospice utilization for the previous calendar year. The intervention period was from June 1, 2020, to December 30, 2020, and included oncologists from both the solid and hematologic malignancies programs. The primary outcome was the proportion of patients between groups with short hospice length of stay (LOS; defined as ≤ 7 days) after 6 months. Secondary outcomes included hospice referral rate, enrollment rate, and median LOS. RESULTS: Forty-seven oncologists participated. The percent of patients with a short hospice stay in the intervention group was lower (17.4%) compared with patients treated by physicians in the usual care group (46.3%, difference = 21.8%; 95% CI, 16.0 to 41.6; P < .001). Receipt of peer comparisons was associated with a greater likelihood of enrolling in hospice (73.7% v 42.8%; difference = 31.1%; 95% CI, 20.4 to 41.7; P < .001) and a longer hospice LOS (37.2 v 18.3 days; difference = 17.2; 95% CI, 8.8 to 25.7 days; P < .001). CONCLUSION: Peer comparisons improved hospice utilization metrics among a group of academic oncologists. Behavioral science offers one pragmatic strategy to overcome suboptimal oncologist decision-making biases related to hospice use.

show abstract

“…Even when structured data fields exist, previous research demonstrate that clinical information, such as tumor stage, may be routinely documented as unstructured free text or may not be documented at all. 26,27 Moreover, while cancer registrars are highly trained in abstracting oncology data, there may be discrepancies in documentation that make reporting challenging. 28 Given the large number of complex data elements and patients captured in national cancer registries, complete documentation and abstraction of all data is likely an impracticable task.…”

Section: Discussionmentioning

confidence: 99%

Comparison of machine learning methods for clinical data imputation among a real-world lung cancer cohort

Yang

Hui

Wang

et al. 2022

Preprint

View full text Add to dashboard Cite

PURPOSE Cancer registries are important sources of real-world data (RWD) that reveal insights into practice patterns and cancer patient outcomes, but the prevalence of missing data can be high. Machine learning (ML) imputation methods can be applied to large RWD sets, but the performance of these approaches within cancer registries is unclear. METHODS We identified non-small cell lung cancer (NSCLC) patients within the National Cancer Database diagnosed in 2014 with complete data in 19 variables of known clinical and prognostic significance. We generated synthetic missing data for each variable, then performed imputation using substitution (control) and five different ML approaches. Imputation efficacy was measured by normalized root-mean-square error (RMSE) for continuous variables and proportion of falsely classified entries (PFC) for categorical variables. We also measured algorithm runtimes and the impact of incorporating imputed values on survival modeling. RESULTS 50,790 NSCLC patients were included for this study, with 81 features for each patient after data preprocessing. Among the tested ML methods, SoftImpute had the lowest RMSE (best performance) for continuous variables ranging from 0.071 to 0.080 for 10% to 50% missing data, and MissForest had the lowest PFC (best performance) for categorical variables ranging from 0.251 to 0.311 for 10 to 50% missing data. SoftImpute had a runtime of 3.28x10-4 seconds per patient record, and MissForest averaged 2.96x10-3 seconds per patient record. Deep learning imputation using a denoising autoencoder did not achieve improved performance despite higher algorithm runtimes. Cox models incorporating ML imputed data achieved similar C-index ranging from 0.787 to 0.801 for all ML methods tested. CONCLUSION ML imputation achieved promising performance for NSCLC patients within a large national cancer registry.

show abstract

Association of Peer Comparison Emails With Electronic Health Record Documentation of Cancer Stage by Oncologists

Cited by 11 publications

References 18 publications

Chasm Between Cancer Quality Measures and Electronic Health Record Data Quality

Chasm Between Cancer Quality Measures and Electronic Health Record Data Quality

Oncologist Peer Comparisons as a Behavioral Science Strategy to Improve Hospice Utilization

Comparison of machine learning methods for clinical data imputation among a real-world lung cancer cohort

Contact Info

Product

Resources

About