Exploring Robust Methods for Evaluating Treatment and Comparison Groups in Chronic Care Management Programs

Abstract

Evaluation of chronic care management (CCM) programs is necessary to determine the behavioral, clinical, and financial value of the programs. Financial outcomes of members who are exposed to interventions (treatment group) typically are compared to those not exposed (comparison group) in a quasi-experimental study design. However, because member assignment is not randomized, outcomes reported from these designs may be biased or inefficient if study groups are not comparable or balanced prior to analysis. Two matching techniques used to achieve balanced groups are Propensity Score Matching (PSM) and Coarsened Exact Matching (CEM). Unlike PSM, CEM has been shown to yield estimates of causal (program) effects that are lowest in variance and bias for any given sample size. The objective of this case study was to provide a comprehensive comparison of these 2 matching methods within an evaluation of a CCM program administered to a large health plan during a 2-year time period. Descriptive and statistical methods were used to assess the level of balance between comparison and treatment members pre matching. Compared with PSM, CEM retained more members, achieved better balance between matched members, and resulted in a statistically insignificant Wald test statistic for group aggregation. In terms of program performance, the results showed an overall higher medical cost savings among treatment members matched using CEM compared with those matched using PSM

Achieving Coordinated Care for Patients With Complex Cases of Cancer: A Multiteam System Approach

“…Quality improvement through both pragmatic trials and implementation research offer opportunity to leverage evidence-based interventions, notably from primary care, 99–105 to understand how team-based approaches to coordination can be leveraged in oncology . Context matters: 106–108 settings like federally-qualified health centers or community clinics outside of vertically-integrated systems like Parkland are important sties to understand how MTS operations can be sustained across clinic organizations and practices, especially absent a unifying, common EMR.…”

An evaluation of exact matching and propensity score methods as applied in a comparative effectiveness study of inhaled corticosteroids in asthma

“…He does not include exact matching, however, and does not investigate the impact of the method choice on the primary endpoint of interest. Studies by Wells et al12 and Fullerton et al13 compare matching methods in real data sets, including exact matching, and discuss the strengths and weaknesses of each. Despite these useful studies, it is important to gain more evidence in this area, to strengthen the conclusions that are drawn from the analyses of one data set and to allow investigators to make more informed decisions on the design of observational studies.…”

Health Care Savings with the Patient-Centered Medical Home: Community Care of North Carolina's Experience

“…More sophisticated, quasi-experimental methods have been recommended to conduct robust evaluations of programs such as CCNC. [32][33] Consistent with those recommendations, this study extends the earlier actuarial research by refining the sample, using pre-post comparison methods, controlling for a number of additional factors, and applying hierarchical regression models to adjust for potential correlation among outcomes.…”