The meaning of vaguely quantified frequency response options on a quality of life scale depends on respondents’ medical status and age

Abstract

Purpose Self-report items in quality of life (QoL) scales commonly use vague quantifiers like “sometimes” or “often” to measure the frequency of health-related experiences. This study examined whether the meaning of such vaguely quantified response options differs depending on people’s medical status and age, which may undermine the validity of QoL group comparisons. Methods Respondents (n = 600) rated the frequency of positive and negative QoL experiences using vague quantifiers (never, rarely, sometimes, often, always) and provided open-ended numeric frequency counts for the same items. Negative binomial regression analyses examined whether the numeric frequencies associated with each vague quantifier differed between medical status (no vs. one or more medical conditions) and age (18–40 years vs. 60+ years) groups. Results Compared to respondents without a chronic condition, those with a medical condition assigned a higher numeric frequency to the same vague quantifiers for negative QoL experiences; this effect was not evident for positive QoL experiences. Older respondents’ numeric frequencies were more extreme (i.e., lower at the low end and somewhat higher at the high end of the response range) than those of younger respondents. After adjusting for these effects, differences in QoL became somewhat more pronounced between medical status groups, but not between age groups. Conclusions The results suggest that people with different medical backgrounds and age do not interpret vague frequency quantifiers on a QoL scale in the same way. Open-ended numeric frequency reports may be useful to detect and potentially correct for differences in the meaning of vague quantifiers.

To bin or not to bin? A comparison of symptom frequency response formats in the assessment of health-related quality of life

“…This is particularly true for the vague quantifiers that tend to be associated with moderate to severe symptoms ("Sometimes" and "Often"). These findings are consistent with prior research suggesting that people may interpret vague quantifiers differently based on individual differences [5].…”

“…The authors fit the mixture IRT model to HDSM data from the BRFSS and found that nearly one-third of respondents engaged in some type of estimation behavior, only endorsing multiple-of-five responses. When considered with previous research showing difficulties many participants experience with exact recall [5,20,27], these findings bring to light some of the challenges associated with using open-ended frequencies for retrospective assessment. The benefits, and costs, this type of response format may offer in terms of measuring individual differences remain unclear.…”

“…Most notably, people have a tendency to report values that are multiples of five, a phenomenon known as heaping [18]. For example, it is more common for people to report experiencing a symptom for 10 days than 9 or 11 days, consistent with the theory that some respondents use cognitive heuristics to estimate frequencies rather than counting over longer recall periods [5,[19][20][21][22]. Heaping makes the psychometric analysis of multivariate open-ended frequencies particularly challenging, as the responses do not tend to follow a standard count distribution that can be easily implemented in conventional latent variable software [16].…”

“…An exact number of days provides a concrete measure of time that is easily understood by policymakers and, due to its ratio scale, may be more sensitive to short-term changes in health than other response formats [4]. Further, because frequencies are in non-relative units (e.g., 4 days) rather than subjective units (e.g., Rarely), they are more likely to be interpreted consistently across respondents, facilitating score comparison [5][6][7]. Schneider and Stone found that respondents with a chronic health condition assigned higher open-ended numeric values to the same vague quantifiers than respondents without a chronic health condition [5], likely due in part to the response shift phenomenon in which people with chronic illness positively adapt to their disease and report higher levels of HRQOL, despite having worse health [8,9].…”

The development and validation of the Dementia Quality of Life Scale for Older Family Carers (DQoL-OC)

“…However, these findings suggest that concepts and concerns related to QoL in later life are different from the general population. As such, our findings would suggest that age plays a significant role in the subjective appraisal of QoL as carers may adapt their perception of well-being as dementia progresses (Schneider & Stone, 2016). More thorough statistical investigation of the scale should be carried out in the future to clarify these relationships.…”