The impact of response format on attitude measurement

Response Bias and Reliability in Verbal Agreement Rating Scales: Does Polarity and Verbalization of the Middle Category Matter?

“…The research that does exist has investigated reliability and generated mixed results. Krebs (2012) found a higher Cronbach's a coefficient in the bipolar than in the unipolar scales, irrespective of whether numeric or verbal labels are used. Using a reliability estimation method within the frame of the popular latent variable modeling (LVM) approach (Raykov & Marcoulides, 2011), Menold and Raykov (2015) found a higher reliability for the unipolar application than for the bipolar agreement rating scale.…”

“…A comparison between unipolar and bipolar verbalization is nevertheless sensible because not agreement is often used interchangeably with disagreement (e.g., in international surveys, see, e.g., for the ISSP in the Introduction section) and gathering empirical evidence on their comparability is essential for such a use. The unipolar and bipolar rating scales were realized in the tradition of the corresponding previous research (Krebs, 2012;Schwarz et al, 1991), which was important for the comparison of the results.…”

“…In German, a simplification "partly/partly" ("teils-teils") has usually been used. Unlike the meaning of indifference or ambivalence in bipolar rating scales, the middle category of a unipolar rating scale is associated with a moderate position or degree that can be articulated by the verbal labels "moderate," "fair," "middle," "in the middle" (Rohrmann, 1978), "to some extent" (Krebs, 2012), or "about a half" (Saris et al, 2010).…”

“…Unlike bipolar rating scales, unipolar rating scales consist of one single rating spectrum, for example, agreement, application, or liking. A popular bipolar "disagree/ agree" rating scale graduates disagreement as one and agreement as the other continuum (Krebs, 2012), whereas its unipolar "do not agree/agree" version only graduates degrees of agreement. The verbalization of the middle category relates to the scale polarity because its meaning is different in bipolar and unipolar rating scales (Krosnick & Fabrigar, 1997;Schaeffer & Presser, 2003).…”

“…Previous research on the polarity of rating scales used face-to-face interviews (O'Muircheartaigh, Gaskell, & Wright, 1995;Schwarz, Knäuper, Hippler, Noelle-Neumann, & Clark, 1991) or selfadministered paper questionnaires (Krebs, 2012;Schwarz & Hippler, 1994) rather than online data collection. The research presented in this article is therefore conducted with the additional aim of providing insights for online survey research.…”