Using a retrospective pretest instead of a conventional pretest is replacing biases: a qualitative study of cognitive processes underlying responses to thentest items

Abstract

Background The thentest design aims to detect and control for recalibration response shift. This design assumes (1) more consistency in the content of the cognitive processes underlying patients’ quality of life (QoL) between posttest and thentest assessments than between posttest and pretest assessments; and (2) consistency in the time frame and description of functioning referenced at pretest and thentest. Our objective is to utilize cognitive interviewing to qualitatively examine both assumptions.MethodsWe conducted think-aloud interviews with 24 patients with cancer prior to and after radiotherapy to elicit cognitive processes underlying their assessment of seven EORTC QLQ-C30 items at pretest, posttest and thentest. We used an analytic scheme based on the cognitive process models of Tourangeau et al. and Rapkin and Schwartz that yielded five cognitive processes. We subsequently used this input for quantitative analysis of count data.ResultsContrary to expectation, the number of dissimilar cognitive processes between posttest and thentest was generally larger than between pretest and posttest across patients. Further, patients considered a range of time frames when answering the thentest questions. Moreover, patients’ description at the thentest of their pretest functioning was often not similar to that which was noted at pretest. Items referring to trouble taking a short walk, overall health and QoL were most often violating the assumptions.ConclusionsBoth assumptions underlying the thentest design appear not to be supported by the patients’ cognitive processes. Replacing the conventional pretest–posttest design with the thentest design may simply be replacing one set of biases with another.Electronic supplementary materialThe online version of this article (doi:10.1007/s11136-015-1175-4) contains supplementary material, which is available to authorized users.

Declawing the dinosaurs in the science classroom: Reducing Christian teachers’ anxiety and increasing their efficacy for teaching evolution

“…On the other hand, the retrospective design introduces validity threats of its own resulting from, for example, faulty recall and social desirability biases (Hill & Betz, ). Thus, choosing between the two methods results in an unavoidable tradeoff (Taminiau‐Bloem et al, ). Because we were interested in teachers’ subjective impressions about the effectiveness of the workshop vis à vis their emotions and confidence, we chose the retrospective design in addition to the traditional pretest.…”

Is there a response shift in generic health-related quality of life 6 months after glioma surgery?

“…The underlying assumptions of the then-test used in this study have recently been criticised [27]. Most importantly, the assumption of cognitive consistency of respondents at the different time points may not hold true [27].…”

“…Most importantly, the assumption of cognitive consistency of respondents at the different time points may not hold true [27]. Also, recall bias of previous health condition is a concern [25, 27], especially since this may be more pronounced in patients with cognitive deficits [1].…”

“…Response shift refers to the phenomenon where patients score better because over time they adapt to a new situation; for instance, a handicap or an illness [17, 18]. It is considered to involve elements of recalibration of internal standards, change in priorities and/or a different view upon the concept of HRQoL [3, 8, 27]. In patients with gliomas all these elements may change when faced with neurological deficits or being diagnosed with a life-threatening illness [18].…”

“…The most common approach to study response shift is the so-called “then-test approach” where respondents retrospectively score how they at time of follow-up consider their earlier or baseline HRQoL in the light of their new situation [17, 25, 27]. …”