Are Comparisons of Consumer Satisfaction With Providers Biased by Nonresponse or Case-Mix Differences?

Simon, Gregory E.; Rutter, Carolyn M.; Crosier, Marlan; Scott, Jennifer; Operskalski, Belinda H.; Ludman, Evette

doi:10.1176/ps.2009.60.1.67

Cited by 12 publications

(4 citation statements)

References 21 publications

(13 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Quantitative measures of patient satisfaction are sometimes used to evaluate therapists, and these may even include a brief measure of patient outcome (e.g., one question asking the patient if they felt improved). However, satisfaction measures are typically only given to a subset of patients and are notoriously skewed (e.g., most patients report very high satisfaction) and not uniformly related to treatment outcomes (Imel, Hubbard, Rutter, & Simon, 2013; Simon et al, 2009; Simon, Imel, Ludman, & Steinfeld, 2012). As an exception to the rule, Lambert and colleagues have designed a feedback system where outcomes are regularly tracked, and each patient’s progress is compared with a predicted treatment response curve.…”

Section: How Are Therapists Currently Evaluated?mentioning

confidence: 99%

Removing very low-performing therapists: A simulation of performance-based retention in psychotherapy.

Imel¹,

Sheng²,

Baldwin³

et al. 2015

Psychotherapy

View full text Add to dashboard Cite

Therapists can impact the likelihood a given patient will benefit from psychotherapy. However, therapists are rarely held accountable for their patients' outcomes. As a result, low performing providers likely continue to practice alongside providers with high response rates. In the current study, we conducted a Monte Carlo simulation to illustrate a thought experiment—what happens to patient outcomes if therapists with the worst outcomes were removed from practice? We drew initial samples of 50 therapists from three simulated populations of 1,000 therapists with a mean patient response rate of 50% and different effect sizes for therapist variability in outcomes. We simulated 30 patient outcomes for each therapist, with outcome defined as response to treatment versus no response. We removed therapists with response rates in the bottom 5% and replaced them with a random sample of therapists from the population. Over 10 years, the difference in responses between the lowest and highest performing therapists was substantial (between 697 and 997 additional responses to treatment). After repeatedly removing the lowest performing providers 40 times (simulating a 10 year time span), response rates increased substantially. The cumulative number of patient responses (i.e., summing the total number of responses across 10 years) increased by 4266, 6404, and 9307 when therapists accounted for 5%, 10%, or 20% of the patient outcome variance, respectively. These findings indicate that performance-based retention of therapists could improve the quality of psychotherapy in health systems by improving the average response rate and decreasing the probability that a patient will be treated by a therapist who has little chance of helping.

show abstract

Section: How Are Therapists Currently Evaluated?mentioning

confidence: 99%

Removing very low-performing therapists: A simulation of performance-based retention in psychotherapy.

Imel¹,

Sheng²,

Baldwin³

et al. 2015

Psychotherapy

View full text Add to dashboard Cite

show abstract

“…It has been proposed that patients who are not satisfied with care, or who had a poor outcome, may be less likely to respond to surveys regarding patient satisfaction (Mazor et al, 2002), leading to response bias. Although recent work has questioned this premise (Simon et al, 2009), we acknowledge that we are only able to make conclusions regarding those subjects who responded to our survey. It should be noted that our response rate of 13.6% is arguably low and suggests the potential for response bias.…”

Section: Discussionmentioning

confidence: 91%

The relationship between patient satisfaction with physical therapy care and global rating of change reported by patients receiving worker's compensation

Beattie¹,

Nelson

Heintzelman

2010

Physiotherapy Theory and Practice

View full text Add to dashboard Cite

This study examined relationships between patient satisfaction with physical therapy care and global rating of change; 1,944 respondents completed the Medrisk Instrument for Measuring Patient Satisfaction with Physical Therapy Care (MR-12) and a nine-point Global Rating of Change Scale (GROC) following a course of physical therapy for a work-related musculoskeletal problem. Ninety percent of all respondents reported being satisfied or very satisfied with their overall care, whereas 70.1% of all respondents indicated they improved following treatment. Respondents who reported improvement had significantly higher scores for all measures of satisfaction (p<0.01) than did those who reported failure to improve; however, both of these groups had mean scores of greater than 4.0 on the MR-12, indicating that respondents were likely to be satisfied or very satisfied with care regardless of perceived change following treatment. Scores>4.0 from the MR-12 had high sensitivity to detect those respondents classified as "improved" (0.87-0.95), but low specificity to differentiate between those who were classified as "improved" and those who were classified as "did not improve" (0.22-0.30). Our findings support the hypothesis that patient satisfaction with care is primarily independent of perceived clinical change.

show abstract

“…Patient and consumer survey responses about experiences, quality, and satisfaction correlate with personal characteristics like health, education, and age, even among people with the same plan or provider Simon et al 2009;Rahmqvist and Bara 2010). These associations combine reporting tendencies with real differences in experiences.…”

mentioning

confidence: 95%

“…; Simon et al. ; Rahmqvist and Bara ). These associations combine reporting tendencies with real differences in experiences.…”

mentioning

confidence: 97%

Implications of Variation in the Relationships between Beneficiary Characteristics and Medicare Advantage CAHPS Measures

Hatfield

Zaslavsky

2016

Health Services Research

View full text Add to dashboard Cite

show abstract

Are Comparisons of Consumer Satisfaction With Providers Biased by Nonresponse or Case-Mix Differences?

Cited by 12 publications

References 21 publications

Removing very low-performing therapists: A simulation of performance-based retention in psychotherapy.

Removing very low-performing therapists: A simulation of performance-based retention in psychotherapy.

The relationship between patient satisfaction with physical therapy care and global rating of change reported by patients receiving worker's compensation

Implications of Variation in the Relationships between Beneficiary Characteristics and Medicare Advantage CAHPS Measures

Contact Info

Product

Resources

About