Avoiding Randomization Failure in Program Evaluation, with Application to the Medicare Health Support Program

King, Gary; Nielsen, Richard A.; Coberley, Carter; Pope, James E.; Wells, Aaron

doi:10.1089/pop.2010.0074

Cited by 36 publications

(30 citation statements)

References 15 publications

(13 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As we cannot ascertain the order of the cessation aids, nor whether use of an additional aid was influenced by e-cigarette use or some other baseline characteristic (such as nicotine dependence), we chose to include models with (Models 3 and 5) and without (Models 2 and 4) use of other cessation aids, acknowledging the results from Models 3 and 5 may be sensitive to posttreatment bias. 41 Future observational studies should include items about the most recent quit attempt to ascertain the effect of an aid or combination of aids on the subsequent attempt. Entropy balancing has some inherent limitations that deserve discussion.…”

Section: Discussionmentioning

confidence: 99%

E-Cigarettes and Smoking Cessation: Insights and Cautions From a Secondary Analysis of Data From a Study of Online Treatment-Seeking Smokers

Pearson¹,

Stanton²,

Cha³

et al. 2014

NICTOB

View full text Add to dashboard Cite

Background: Evidence from observational studies regarding the association between electronic cigarette (e-cigarette) use and cessation is mixed and difficult to interpret. Utilizing 2 analytic methods, this study illustrates challenges common in analyses of observational data, highlights measurement challenges, and reports associations between e-cigarette use and smoking cessation. Methods: Data were drawn from an ongoing web-based smoking cessation trial. The sample was comprised of 2,123 participants with complete 3-month follow-up data. Logistic regression models with and without entropy balancing to control for confounds were conducted to evaluate the association between e-cigarette use and 30-day cigarette smoking abstinence. Results: At follow-up, 31.7% of participants reported using e-cigarettes to quit in the past 3 months. E-cigarette users differed from nonusers on baseline characteristics including cigarettes per day, Fagerström score, quit attempt in the past year, and previous use of e-cigarettes to quit. At followup, e-cigarette users made more quit attempts and employed more cessation aids than smokers who did not use e-cigarettes to quit. E-cigarette use was negatively associated with abstinence after adjustment for baseline characteristics; however, the association was not significant after additional adjustment for use of other cessation aids at 3 months. Conclusions: The magnitude and significance of the estimated association between e-cigarette use and cessation in this study were dependent upon the analytical approach. Observational studies should employ multiple analytic approaches to address threats to validity. Future research should employ better measures of patterns of and reasons for e-cigarette use, frequency of e-cigarette use, and concurrent use of cessation aids.

show abstract

Section: Discussionmentioning

confidence: 99%

E-Cigarettes and Smoking Cessation: Insights and Cautions From a Secondary Analysis of Data From a Study of Online Treatment-Seeking Smokers

Pearson¹,

Stanton²,

Cha³

et al. 2014

NICTOB

View full text Add to dashboard Cite

show abstract

“…The healthy controls were also divided into four groups that matched the average age of each of the patient groups: group 1: n = 8, mean age 31.5 ± 2.0, four females, group 2: n = 10, mean age 32.5 ± 4.2, six females, group 3: n = 10, mean age 36.1 ± 6.8, seven females, group 4: n = 12, mean age 38.8 ± 8.7, eight females. The stratification was done using the method of nearest neighbor matching with replacement (King, Nielsen, Coberley, Pope, & Wells, ; Stuart, ) of type N + 4:N (Magnoni et al, ).…”

Section: Methodsmentioning

confidence: 99%

Changes in structural network are associated with cortical demyelination in early multiple sclerosis

Mangeat

Badji

Ouellette

et al. 2018

Human Brain Mapping

View full text Add to dashboard Cite

The aim of this study was to investigate the interplay between structural connectivity and cortical demyelination in early multiple sclerosis. About 27 multiple sclerosis patients and 18 age-matched controls underwent two MRI scanning sessions. The first was done at 7T and involved acquiring quantitative T and T * high-resolution maps to estimate cortical myelination. The second was done on a Connectom scanner and consisted of acquiring high angular resolution diffusion-weighted images to compute white matter structural connectivity metrics: strength, clustering and local efficiency. To further investigate the interplay between structural connectivity and cortical demyelination, patients were divided into four groups according to disease-duration: 0-1 year, 1-2 years, 2-3 years, and >3 years. ANOVA and Spearman's correlations were used to highlight relations between metrics. ANOVA detected a significant effect between disease duration and both cortical myelin (p = 2 × 10 ) and connectivity metrics (p < 10 ). We observed significant cortical myelin loss in the shorter disease-duration cohorts (0-1 year, p = .0015), and an increase in connectivity in the longer disease-duration cohort (2-3 years, strength: p = .01, local efficiency: p = .002, clustering: p = .001). Moreover, significant covariations between myelin estimation and white matter connectivity metrics were observed: Spearman's Rho correlation coefficients of 0.52 (p = .0003), 0.55 (p = .0001), and 0.53 (p = .0001) for strength, local efficiency, and clustering, respectively. An association between cortical myelin loss and changes in white matter connectivity in early multiple sclerosis was detected. These changes in network organization might be the result of compensatory mechanisms in response to the ongoing cortical diffuse damage in the early stages of multiple sclerosis.

show abstract

“…Additionally, random assignment does not necessarily guarantee equivalent groups, as evidenced in the randomized block design employed in the recent Medicare Health Support pilots. 3,4 In these cases, the designs did not achieve equal distribution of members with similar characteristics of age, sex, race, mortality likelihood, and medical expenditures. Therefore, it is possible to randomly assign individuals, whether using complete or block randomization, to treatment and control and yet still have groups that are not equivalent or comparable.…”

Section: Historical Ccm Study Designsmentioning

confidence: 97%

“…Based on the Care Continuum Alliance (CCA) guidelines, incorporation of a valid study design is essential to (1) ascertain the value of disease management (DM) programs (referenced as CCM in this article) in achieving favorable outcomes for populations with chronic disease, (2) further improve the delivery of DM services and foster competition among industry participants based on objective criteria, (3) meaningfully advance the delivery of health care services through the reporting and dissemination of interventions that reduce the burden of disease, and (4) help assess which health plans or delivery systems are providing higher quality care at a more affordable cost. 2 The CCA recommends the use of a randomized controlled study design to assess causality of DM in achieving outcomes.…”

Section: Historical Ccm Study Designsmentioning

confidence: 99%

Exploring Robust Methods for Evaluating Treatment and Comparison Groups in Chronic Care Management Programs

Wells

Hamar

Bradley

et al. 2013

Population Health Management

Self Cite

View full text Add to dashboard Cite

Evaluation of chronic care management (CCM) programs is necessary to determine the behavioral, clinical, and financial value of the programs. Financial outcomes of members who are exposed to interventions (treatment group) typically are compared to those not exposed (comparison group) in a quasi-experimental study design. However, because member assignment is not randomized, outcomes reported from these designs may be biased or inefficient if study groups are not comparable or balanced prior to analysis. Two matching techniques used to achieve balanced groups are Propensity Score Matching (PSM) and Coarsened Exact Matching (CEM). Unlike PSM, CEM has been shown to yield estimates of causal (program) effects that are lowest in variance and bias for any given sample size. The objective of this case study was to provide a comprehensive comparison of these 2 matching methods within an evaluation of a CCM program administered to a large health plan during a 2-year time period. Descriptive and statistical methods were used to assess the level of balance between comparison and treatment members pre matching. Compared with PSM, CEM retained more members, achieved better balance between matched members, and resulted in a statistically insignificant Wald test statistic for group aggregation. In terms of program performance, the results showed an overall higher medical cost savings among treatment members matched using CEM compared with those matched using PSM

show abstract

Avoiding Randomization Failure in Program Evaluation, with Application to the Medicare Health Support Program

Cited by 36 publications

References 15 publications

E-Cigarettes and Smoking Cessation: Insights and Cautions From a Secondary Analysis of Data From a Study of Online Treatment-Seeking Smokers

E-Cigarettes and Smoking Cessation: Insights and Cautions From a Secondary Analysis of Data From a Study of Online Treatment-Seeking Smokers

Changes in structural network are associated with cortical demyelination in early multiple sclerosis

Exploring Robust Methods for Evaluating Treatment and Comparison Groups in Chronic Care Management Programs

Contact Info

Product

Resources

About