Listening to the elephant in the room: response-shift effects in clinical trials research

Abstract

Background While a substantial body of work postulates that adaptation (response-shift effects) may serve to hide intervention benefits, much of the research was conducted in observational studies, not randomized-controlled trials. This scoping review identified all clinical trials that addressed response shift phenomena, and characterized how response-shift effects impacted trial findings. Methods A scoping review was done of the medical literature from 1968 to 2021 using as keywords “response shift” and “clinical trial.” Articles were included if they were a clinical trial that explicitly examined response-shift effects; and excluded if they were not a clinical trial, a full report, or if response shift was mentioned only in the discussion. Clinical-trials papers were then reviewed and retained in the scoping review if they focused on randomized participants, showed clear examples of response shift, and used reliable and valid response-shift detection methods. A synthesis of review results further characterized the articles’ design characteristics, samples, interventions, statistical power, and impact of response-shift adjustment on treatment effect. Results The search yielded 2148 unique references, 25 of which were randomized-controlled clinical trials that addressed response-shift effects; 17 of which were retained after applying exclusion criteria; 10 of which were adequately powered; and 7 of which revealed clinically-important response-shift effects that made the intervention look significantly better. Conclusions These findings supported the presumption that response shift phenomena obfuscate treatment benefits, and revealed a greater intervention effect after integrating response-shift related changes. The formal consideration of response-shift effects in clinical trials research will thus not only improve estimation of treatment effects, but will also integrate the inherent healing process of treatments. Key points This scoping review supported the presumption that response shift phenomena obfuscate treatment benefits and revealed a greater intervention effect after integrating response-shift related changes. The formal consideration of response-shift effects in clinical trials research will not only improve estimation of treatment effects but will also integrate the inherent healing process of treatments.

Response‐shift effects in childhood cancer survivors: A prospective study

“…A recent scoping review of clinical trials suggests that ignoring response‐shift effects will likely mislead the interpretation for QOL score change over time 30 . Building on this scoping review as well as our study results, it would be worthwhile to consider response‐shift effects in clinical decision‐making.…”

“…A recent scoping review of clinical trials suggests that ignoring response-shift effects will likely mislead the interpretation for QOL score change over time. 30 Building on this scoping review as well as our study results, it would be worthwhile to consider response-shift effects in clinical decision-making. It is clear that without such consideration the current model of repeated QOL assessment in monitoring cancer late-effect progression is missing important information about what matters to patients as their context changes.…”

Clinical trial evidence of quality-of-life effects of disease-modifying therapies for multiple sclerosis: a systematic analysis

“…Fourth, QoL measurements are often measured longitudinally with multiple time points, yet we only extracted and assessed the measurement with the longest follow-up available. QoL measurements may also be prone to response shift, whereby the individual’s reference of well-being changes overtime [ 61 ], potentially impacting estimation of QoL in clinical trials with longitudinal follow-up [ 62 ].…”

Longitudinal invariance of the Patient Health Questionnaire–9 among patients receiving pharmacotherapy for major depressive disorder: A secondary analysis of clinical trial data.

“…While response shifts, such as an enhanced understanding and awareness of depressive symptoms, may be a critical part of treatment (Fokkema et al, 2013), the resultant lack of LMI has serious implications for how measure scores are interpreted. There is evidence that failing to account for a violation of LMI in analyses can mask treatment effects (Schwartz et al, 2022) and increase the risk of Type I and Type II errors in research studies (Chen & Yang, 2020; Georgeson et al, 2022). In clinical practice, a lack of LMI may result in over- or underestimation of symptom change (or even the erroneous assumption of deterioration), thereby interfering with treatment planning and monitoring (Sawatzky et al, 2021).…”