Which Scale Direction is More Difficult for Respondents to Use? An Eye-tracking Study

Abstract

Scale direction is found to affect response distributions by yielding more selections of scale points closer to the beginning of the scale. Although this phenomenon has been empirically demonstrated in different modes of data collection for respondents with different demographic characteristics, it is not clear which scale direction is cognitively difficult for respondents to use. Eye-tracking is used to address this research question because it provides a direct window into how respondents process survey questions. I compared dilation and fixation measures across scale direction and found that the satisfaction scale and the frequency scale running from “Never” to “Very Often” had more and longer fixations. In addition, the dissatisfaction options incurred larger dilations, more peak dilations, and more and longer fixations than the other parts of the scale. Findings of this paper shed light on how respondents process response scales and have practical implications for questionnaire design. Furthermore, findings of this paper demonstrate the utility and potential of using eye-tracking in general and dilation measures in particular to understand cognitive burden of answering survey questions.