Response tendencies due to item wording using eye-tracking methodology accounting for individual differences and item characteristics

Koutsogiorgi, Chrystalla; Michaelides, Michalis P.

doi:10.3758/s13428-021-01719-x

Cited by 8 publications

(7 citation statements)

References 96 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Respondents responded slower to NWI. This is in agreement with other studies investigating response latency (Koutsogiorgi & Michaelides, 2022; Swain et al, 2008). However, because the four items of RSES were not experimentally controlled for length and context, this finding should be interpreted with caution.…”

Section: Discussionsupporting

confidence: 94%

“…Data from 7 additional volunteers were not included in the analysis due to difficulties in vision and/or movement; 10 volunteers were also not included because of large eye movement deviations that were observed during the calibration process. Background data from the same participants were used in an experimental study published by Koutsogiorgi and Michaelides (2022). In the current study, we analyze original data from the RSES as dependent variables.…”

Section: Methodsmentioning

confidence: 99%

“…Hypothesis 3 (H3) : Respondents with higher neuroticism would show greater processing of NWI. This would be expressed with longer times of viewing, more fixations, and more revisits to the negatively valenced words, but lower response times, since the preoccupation of these individuals with the negatively valenced words is anticipated to facilitate later components of responding (Koutsogiorgi & Michaelides, 2022). They may also show greater endorsement of NWI.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Response Tendencies to Positively and Negatively Worded Items of the Rosenberg Self-Esteem Scale With Eye-Tracking Methodology

Koutsogiorgi

Michaelides

2023

European Journal of Psychological Assessment

Self Cite

View full text Add to dashboard Cite

The Rosenberg Self-Esteem Scale (RSES) was developed as a unitary scale to assess attitudes toward the self. Previous studies have shown differences in responses and psychometric indices between the positively and negatively worded items, suggesting differential processing of responses. The current study examined differences in response behaviors toward two positively and two negatively worded items of the RSES with eye-tracking methodology and explored whether those differences were more pronounced among individuals with higher neuroticism, controlling for verbal abilities and mood. Eighty-seven university students completed a computerized version of the scale, while their responses, response time, and eye movements were recorded through the Gazepoint GP3 HD eye-tracker. In linear mixed-effects models, two negatively worded items elicited higher scores (elicited stronger disagreement) in self-esteem, and different response processes, for example, longer viewing times, than two positively worded items. Neuroticism predicted lower responses and more revisits to item statements. Eye-tracking can enhance the examination of response tendencies and the role of wording and its interaction with individual characteristics at different stages of the response process.

show abstract

Section: Discussionsupporting

confidence: 94%

Section: Methodsmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Response Tendencies to Positively and Negatively Worded Items of the Rosenberg Self-Esteem Scale With Eye-Tracking Methodology

Koutsogiorgi

Michaelides

2023

European Journal of Psychological Assessment

Self Cite

View full text Add to dashboard Cite

show abstract

Section: Discussionmentioning

confidence: 99%

The ephemeral nature of wording effects.

Ponce¹,

Irribarra²,

Vergés³

et al. 2023

Journal of Personality and Social Psychology

View full text Add to dashboard Cite

This article explores the analysis and interpretation of wording effects associated with using direct and reverse items in psychological assessment. Previous research using bifactor models has suggested a substantive nature of this effect. The present study uses mixture modeling to systematically examine an alternative hypothesis and surpass recognized limitations in the bifactor modeling approach. In preliminary supplemental Studies S1 and S2, we examined the presence of participants who exhibited wording effects and evaluated their impact on the dimensionality of Rosenberg's Self-Esteem and the Revised Life Orientation Test, confirming the ubiquity of wording effects in scales containing direct and reverse items. Then, after analyzing the data for both scales (n = 5,953), we found that, despite a significant association between wording factors (Study 1), a low proportion of participants simultaneously exhibited asymmetric responses in both scales (Study 2). Similarly, despite finding both longitudinal invariance and temporal stability of this effect in three waves (n = 3,712, Study 3), a small proportion of participants was identified with asymmetric responses over time (Study 4), reflected in lower transition parameters compared to the other patterns of profiles examined. In both cases, we illustrate how bifactor models capitalize on the responses of individuals who do not even exhibit wording effects, yielding spurious correlations suggesting a substantive nature of the wording effect. These findings support the notion of an ephemeral nature underlying wording effects. The discussion focuses on alternative hypotheses to understand these findings and emphasizes the utility of including reverse items in psychological assessment.

show abstract

“…Our analyses of the scales of self-esteem and dispositional optimism, using mixture modeling, allowed us to identify this pattern of responses and examine, using removal strategies, its impact on the instrument's psychometric properties, among other characteristics. On the other hand, several authors have proposed different approaches to detect careless responses with scales that contain or do not contain reverse items, including attentional evaluation techniques (bogus or instructed response items; Meade & Craig, 2012;Oppenheimer et al, 2009), evaluation of auxiliary data or paradata (response latencies or fixations; Henninger & Plieninger, 2021;Koutsogiorgi & Michaelides, 2022;Zhang & Conrad, 2014) and detection of outliers or response functions (Curran et al, 2016). However, the evidence shows a series of drawbacks associated with the use and interpretation of these techniques, including the lack of effectiveness and consistency between them due to the arbitrariness with which their cutoff points are defined, depending on the dataset used (Curran, 2016;Nissen et al, 2016) and low sensitivity and specificity for the detection of not completely random response patterns (Meade et al, 2017;Schroeders et al, 2022).…”

Section: To Reverse or Not To Reverse Items?mentioning

confidence: 99%

The ephemeral nature of wording effects

Ponce¹,

Irribarra²,

Vergés³

et al. 2022

Preprint

View full text Add to dashboard Cite

This article examines the analysis and interpretation of wording effects associated with using direct and reverse items in psychological assessment. Previous research using bifactor models has suggested a substantive nature of this effect. The present study uses mixture modeling to systematically examine an alternative hypothesis to surpass acknowledged limitations in the bifactor modeling approach. In Studies 1 and 2, we examined the presence of participants who exhibited wording effects and evaluated their impact on the dimensionality of Rosenberg’s Self-Esteem (RSES) and the Revised Life Orientation Test (LOT-R), confirming the ubiquity of wording effects in scales containing direct and reverse items. Then, using the data for both scales (n = 5953), we found that while there was a significant association between wording factors (Study 3a), there was also a low proportion of participants simultaneously exhibiting asymmetric responses in both scales (Study 3b). Similarly, despite finding both longitudinal invariance and temporal stability of this effect in three waves (n = 3712, Study 4a), the proportion of participants consistently identified with symmetric responses was low over time (Study 4b), reflected in lower transition parameters compared to the other patterns of profiles examined. These findings provide strong support for the notion of an ephemeral nature underlying wording effects. Discussion focuses on alternative hypotheses to understand these findings and emphasizes, contrary to popular misconceptions, the utility of including inverse items in psychological assessment.

show abstract

Response tendencies due to item wording using eye-tracking methodology accounting for individual differences and item characteristics

Cited by 8 publications

References 96 publications

Response Tendencies to Positively and Negatively Worded Items of the Rosenberg Self-Esteem Scale With Eye-Tracking Methodology

Response Tendencies to Positively and Negatively Worded Items of the Rosenberg Self-Esteem Scale With Eye-Tracking Methodology

The ephemeral nature of wording effects.

The ephemeral nature of wording effects

Contact Info

Product

Resources

About