Assessing Three Sources of Misresponse to Reversed Likert Items

Abstract

Data collected through multi-item Likert scales that contain reversed items often exhibit problems, such as unexpected factor structures and diminished scale reliabilities. These problems arise when respondents select responses on the same side of the scale neutral point for both reversed and nonreversed items, a phenomenon the authors call "misresponse." Across four experiments and an exploratory study using published data, the authors find that misresponse to reversed Likert items averaged approximately 20%, twice the level identified as problematic in previous simulation studies. Counter to prevailing thought, the patterns of misresponse and response latency across manipulated items could not be attributed to respondent inattention or acquiescence. Instead, the pattern supports an item verification difficulty explanation, which holds that task complexity, and thus misresponse and response latency, increases with the number of cognitive operations required for a respondent to compare a scale item with his or her belief. The observed results are well explained by the constituent comparison model.

Perceived organizational support and organizational identification: joint moderating effects of employee exchange ideology and employee investment

“…Exchange ideology was measured using eight items from Perceived organizational support and OID 17 Eisenberger et al's (2001) scale. Three reverse-coded items were removed, as they loaded into a separate factor (see e.g., Netemeyer, Bearden and Sharma, 2003;Swain, Weathers and Niedrich, 2008). Prior research has also used only the five main items (e.g., Scott and Colquitt, 2007).…”

Learning from Stakeholder Pressure and Embeddedness: The Roles of Absorptive Capacity in the Corporate Social Responsibility of Dutch Agribusinesses

“…[68] non-market stakeholder pressure scale. To ensure sufficient variability in the questionnaire, the items of this scale were rephrased as interrogative questions [70]. The scale of secondary stakeholder embeddedness consists of six items that measure the extent to which the organization is embedded in networks of environmental organizations in the institutional environment, based on Wuyts and Geyskens' supplier embeddedness scale [69].…”

Measure Twice, Cut down Error: A Process for Enhancing the Validity of Survey Scales

“…For example, rather than developing three separate items to address teachers' enthusiasm, warmth, and communication, we developed one item to address encouragement. To address the issue of the valence of our items, we knew to avoid reverse scored items -especially for younger respondents (see Benson & Hocevar, 1985;Swain, Weathers, & Niedrich, 2008). …”

“…In practice, theoretically ostensible opposites are frequently not arrayed on a continuum (i.e., they are multi-dimensional) and reverse-scored items diminish scale reliability. (Benson & Hocevar, 1985;Cacioppo & Berntson, 1994;Swain, et al, 2008) Use at least 5-7 response anchors For most respondent populations five-point response anchors for unipolar items (that range from a conceptual 0 point to infinity) and seven-point response anchors for bipolar items (that conceptually range from negative infinity to infinity) will work well. (Krosnick, 1999;Weng, 2004) Avoid agree/disagree response anchors Asking respondents to rate their level of agreement to different statements is a cognitively demanding task that increases respondent error and reduces respondent effort in many cases.…”