Decomposition can harm the accuracy of behavioural frequency reports

Abstract

In survey research, the use of decomposition can lead to pronounced reporting errors as seen by overreporting and overall reporting error. A total of 87 subjects answered either decomposed or undecomposed questions concerning telephone calls made by them while at work. The questionnaire conditions varied the length of the reference period (1 week or 6 months), and the type of call (local or long-distance). Decomposition conditions introduced either spatial or temporal cues. In all comparisons, decomposed questions increased overreporting bias relative to undecomposed questions. In addition, undecomposed questions with a 1-week reference period led to increased overreporting bias in comparison to undecomposed/ 6-month questions. Results are consistent with a category split estimation model in which smaller categories are predicted to lead to overreporting, and larger categories to underreporting. Decomposition is not recommended for gaining retrospective reports of nondistinctive, frequent events.

The Science of Asking Questions

“…Checklists appear to reduce omissions, probably because recognition tasks are easier than free recall tasks and because the list structure requires that respondents take more time to process each item. On the other hand, checklists may increase reporting of events that took place before the reference period, and they may lead to overestimates for small categories if the event class is "decomposed" inappropriately (Belli et al 2000, Menon 1997). Thus, a checklist is apt to yield higher overall levels of reporting for a class of events than a single question about the class that includes examples.…”

The Maastricht social participation profile: development and clinimetric properties in older adults with a chronic physical illness

“…Next, following guidelines about questionnaire design [15][16][17][18][19], we constructed three indices, based on the presented definition and using items from the list of social participation examples [6]. We deviated from the definition in two ways.…”

Self‐reported rates of interpersonal conflict vary as a function of questionnaire format: Why age‐related trends in disagreement (and other events) may not be what they seem

“…Cues aid retrieval of salient and irregular events by priming memory and facilitating recall; participants not only remember more events, they remember them more accurately, because they count rather than estimate their occurrence (Menon & Yorkston, 2000). Given the abundance of evidence that cues for recall increase the reporting of events such as the use of the telephone (Belli, Schwarz, Singer, & Tallarico, 2000) and dining at a restaurant (Sudman & Schwarz, 1989), there is good reason to expect similar effects for interpersonal conflict.…”