Effects of Category Order on Answers in Mail and Telephone Surveys<sup>1</sup>

Dillman, Don A.; Brown, Tommy L.; Carlson, John E.; Carpenter, Edwin H.; Lorenz, Frederick O.; Mason, Robert L.; Saltiel, John; Songster, Roberta L.

doi:10.1111/j.1549-0831.1995.tb00600.x

Cited by 33 publications

(12 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Although the findings on primacy effects run counter to findings from previous mail and online computer surveys, as noted earlier by Krosnick and Alwin (1987, p. 202), “many experiments designed to examine response-order effects found none.” Similarly, based on their own experimental research, Dillman et al (1995) found the emergence of significant primacy effects to be exceptional and inconsistent. The absence of response-order effects was also noted by Schwarz, Hippler, and Noelle-Neumann (1991) who state the emergence of response-order effects seems to depend on the interaction of several respondent-level and survey-level factors.…”

Section: Resultsmentioning

confidence: 98%

“…This pattern has been previously uncovered in online computer surveys (Couper et al, 2004; Galesic et al, 2008; Malhotra, 2008) and mail surveys (Krosnick & Alwin, 1987; Schwarz, Hippler, & Noelle-Neumann, 1991), both of which are self-administered surveys with response options presented visually. It should be mentioned, however, that in previous experimental research, primacy effects have been uncovered often but not always (see, e.g., Dillman et al, 1995). Nonetheless, we expected to uncover primacy effects—disproportionately more endorsements for items appearing at the top of the list—and we expected to observe this pattern among both computer respondents and mobile app respondents.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Comparison of Smartphone and Online Computer Survey Administration

Wells

Bailey²,

Link

2013

Social Science Computer Review

View full text Add to dashboard Cite

The dramatic rise of smartphones has profound implications for survey research. Namely, can smartphones become a viable and comparable device for self-administered surveys? The current study is based on approximately 1,500 online U.S. panelists who were smartphone users and who were randomly assigned to the mobile app or online computer mode of a survey. Within the survey, we embedded several experiments that had been previously tested in other modes (mail, PC web, mobile web). First, we test whether responses in the mobile app survey are sensitive to particular experimental manipulations as they are in other modes. Second, we test whether responses collected in the mobile app survey are similar to those collected in the online computer survey. Our mobile survey experiments show that mobile survey responses are sensitive to the presentation of frequency scales and the size of open-ended text boxes, as are responses in other survey modes. Examining responses across modes, we find very limited evidence for mode effects between mobile app and PC web survey administrations. This may open the possibility for multimode (mobile and online computer) surveys, assuming that certain survey design recommendations for mobile surveys are used consistently in both modes.

show abstract

Section: Resultsmentioning

confidence: 98%

Section: Resultsmentioning

confidence: 99%

Comparison of Smartphone and Online Computer Survey Administration

Wells

Bailey²,

Link

2013

Social Science Computer Review

View full text Add to dashboard Cite

show abstract

“…Conversely, the recency effect is expected when options are presented orally (Holbrook et al., ). For example, during interviews, people tend to choose from the last‐offered options (Dillman & Christian, ; Krosnick & Alwin, ; Schwarz & Oyserman, ). Since this article addresses the design of written survey questionnaires to be self‐administered, we will focus on the primacy effect.…”

Section: Response Biases Associated With Response‐order Effectsmentioning

confidence: 99%

Evidence-Based Survey Design: The Use of Ascending or Descending Order of Likert-Type Response Options

Chyung

Kennedy²,

Campbell³

2018

Perf. Improv.

View full text Add to dashboard Cite

Survey designers should be aware of response‐order effects associated with left‐side selection bias, acquiescence bias, and satisficing. A sufficient amount of research has shown that descending‐ordered response scales generate more positive responses from respondents than ascending‐ordered scales. A simple solution to the inflated data obtained from descending‐ordered scales is to present response scales in ascending order. Otherwise, descending‐ordered scales should be used with strategies for reducing response‐order effects.

show abstract

“…. Examples of studies in this vein includeCampbell and Mohr (1950);Becker (1954);Belson (1966);Payne (1971);Carp (1974);Johnson (1981);Chan (1991);Dillman et al (1995); andSchuman and Presser (1996).at AAPOR Institutional Member on April 19, 2010 http://poq.oxfordjournals.org Downloaded from…”

mentioning

confidence: 99%

Order Effects in Complex and Simple Tasks

Malhotra¹

2009

Public Opinion Quarterly

View full text Add to dashboard Cite

There is strong evidence that the order in which response options are presented in surveys significantly affects the answers that respondents provide. According to the theory of survey satisficing, the severity of order effects should increase with task difficulty. However, the tasks provided to respondents in existing studies of response-order effects are generally very simple, making it difficult to evaluate the satisficing hypothesis. Further, evidence from cognitive psychology suggests a completely different mechanism: people are more motivated to persist in completing tasks when they are intricate, challenging, and enriching. I designed survey experiments administered over the Internet consisting of two types of tasks: (1) a complex task in which respondents were asked to rank seven public officials in order of how much they should be blamed for the property damage and loss of life caused by Hurricane Katrina in the city of New Orleans; and (2) a series of simple tasks in which respondents answered items with ordinal response choices on rating scales. I found almost no order effects in the complex task among all educational groups. Conversely, I found significant and substantial order effects in the simple tasks, particularly among low-education respondents. These results suggest that theories of survey satisficing may simplify matters by assuming that satisficing monotonically increases with task difficulty. Moreover, my findings have important implications for questionnaire design, underscoring the importance of randomizing response options. A principal concern of survey methodologists is whether the order in which response options are presented affects the data that respondents produce (Schuman and Presser 1996). In a meta-analysis of several studies of order effects in categorical response choices, Krosnick (1999) and Krosnick and

show abstract

Effects of Category Order on Answers in Mail and Telephone Surveys¹

Cited by 33 publications

References 8 publications

Comparison of Smartphone and Online Computer Survey Administration

Comparison of Smartphone and Online Computer Survey Administration

Evidence-Based Survey Design: The Use of Ascending or Descending Order of Likert-Type Response Options

Order Effects in Complex and Simple Tasks

Contact Info

Product

Resources

About

Effects of Category Order on Answers in Mail and Telephone Surveys1

Cited by 33 publications

References 8 publications

Comparison of Smartphone and Online Computer Survey Administration

Comparison of Smartphone and Online Computer Survey Administration

Evidence-Based Survey Design: The Use of Ascending or Descending Order of Likert-Type Response Options

Order Effects in Complex and Simple Tasks

Contact Info

Product

Resources

About

Effects of Category Order on Answers in Mail and Telephone Surveys¹