Response Tendencies to Positively and Negatively Worded Items of the Rosenberg Self-Esteem Scale With Eye-Tracking Methodology

Abstract

The Rosenberg Self-Esteem Scale (RSES) was developed as a unitary scale to assess attitudes toward the self. Previous studies have shown differences in responses and psychometric indices between the positively and negatively worded items, suggesting differential processing of responses. The current study examined differences in response behaviors toward two positively and two negatively worded items of the RSES with eye-tracking methodology and explored whether those differences were more pronounced among individuals with higher neuroticism, controlling for verbal abilities and mood. Eighty-seven university students completed a computerized version of the scale, while their responses, response time, and eye movements were recorded through the Gazepoint GP3 HD eye-tracker. In linear mixed-effects models, two negatively worded items elicited higher scores (elicited stronger disagreement) in self-esteem, and different response processes, for example, longer viewing times, than two positively worded items. Neuroticism predicted lower responses and more revisits to item statements. Eye-tracking can enhance the examination of response tendencies and the role of wording and its interaction with individual characteristics at different stages of the response process.

Process Data in Computer-Based Assessment

“…For example, Arslan and colleagues (2020) found that even when keeping the item content identical, changing the surface of the item design causes significant changes in process data measures, which highlights the impact of construct-irrelevant design factors on process data. In line with this observation, as demonstrated in the work of Koutsogiorgi and Michaelides (2023) in this special issue, also the poling of items in terms of positive and negative wording can affect the way how items are processed by different respondents, as becomes evident in their eye movement pattern. In the present case, it was a research goal of the authors to investigate attention-related differences towards the different functioning of positive and negative worded items and related response patterns of the established Rosenberg Scale.…”

“…Specifically, behavioral log data from computer-based assessments, including action sequences and pauses (Arslan et al, 2023), time-stamped indicators based on mouse clicks, touch-device input, or keystrokes (Drake et al, 2023;Hahnel et al, 2023;Veerbeek & Vogelaar, 2023), and mouse cursor movements (Pokropek et al, 2023) have been the center of interest in this special issue. Moreover, Koutsogiorgi and Michaelides (2023) analyzed parameters derived from eye movement recordings.…”

“…We believe that we have achieved this ambitious goal by assembling seven excellent research articles in this special issue that illuminate how different types of process data in assessment tasks can be employed in favor of supporting certain diagnostic inferences. Two theoretical groundwork papers (Arslan et al, 2023; Zumbo et al, 2023) and five empirical papers (Drake et al, 2023; Hahnel et al, 2023; Koutsogiorgi & Michaelides, 2023; Pokropek et al, 2023; Veerbeek & Vogelaar, 2023) draw a picture of the crucial role of a strong theoretical foundation and rigorous analysis in making valid inferences from process data, while acknowledging their limitations.…”

“…Contributions focused on cognitive and metacognitive (Arslan et al, 2023; Hahnel et al, 2023; Veerbeek & Vogelaar, 2023; Zumbo et al., 2023) as well as self-regulatory processes (Drake et al, 2023) while individuals worked on assessment tasks. Furthermore, two papers investigated careless response behavior (Pokropek et al, 2023) and response tendencies to positive and negative worded items (Koutsogiorgi & Michaelides, 2023) when answering standardized scales.…”

Short-Scale Construction Using Meta-Analytic Ant Colony Optimization: A Demonstration With the Need For Cognition Scale

“…We also made an open social media call for unpublished studies including the NCS-18 and asked colleagues directly via email to send raw data or the item correlation matrices. Several authors were responsive to our request (Barceló, 2023;Cartwright et al, 2009;Edwards, 2009;Gomes et al, 2013;Grădinaru et al, 2022;Karagiannopoulou et al, 2020;Koutsogiorgi, 2020;Lee et al, 2020;Pilli & Mazzon, 2016;Powell et al, 2016;Sousa et al, 2016;van Tilburg et al, 2019;Weigold & Weigold, 2022;Weng & DeMarree, 2019). This literature search and screening process resulted in 57 publications with 87 samples that were included in our meta-analysis (see Figure 2 for an overview).…”