To Reverse Item Orientation or Not to Reverse Item Orientation, That Is the Question

Abstract

To investigate the effect of using negatively oriented items, we wrote semantic reversals of the items in the Rosenberg Self Esteem Scale, UCLA Loneliness Scale, and the General Belongingness Scale and used them to create four experimental conditions. Participants ( N = 2,019) were recruited through Amazon’s Mechanical Turk. Data were assessed for dimensionality, item functioning, instrument properties, and associations with other variables. Regarding dimensionality, although a two-factor model (positively vs. negatively oriented factors) exhibits better fit than a unidimensional model across all conditions, bifactor indices were used to argue that a unidimensional interpretation of the data can be employed. With respect to item functioning, factor loadings were found to be nearly invariant across conditions, but thresholds were not. Concerning instrument properties, inclusion of negatively oriented items results in lower mean scores and higher score variances. Instruments with both positively and negatively oriented items demonstrated lower reliability estimates than those with only one orientation. For associations with other variables, path coefficients in a model where loneliness mediates the effects of belongingness on life satisfaction and self-esteem were found to vary across conditions. Findings suggest that negatively oriented items have minor impact on instrument quality, but influence measurement model and path coefficients.

Comparing the Psychometric Properties of a Scale Across Three Likert and Three Alternative Formats: An Application to the Rosenberg Self-Esteem Scale

“…Although most of our factor analysis results were consistent with our hypotheses, one unexpected finding was that the Revised-U Likert version had a slightly worse one-factor model fit than the Revised-B Likert version. This finding is the opposite of the results of previous studies (e.g., Dueber et al, 2021; Greenberger et al, 2003; Salazar, 2015; Zhang et al, 2016). Particularly, Greenberger et al’s (2003) study used the RSES in their study and demonstrated that when the original RSES is converted to an unbalanced Likert scale (e.g., all PW items or all NW items), the factor structure becomes unidimensional and consistent with the theoretical factor model.…”

“…As a result, some researchers advocated simply removing NW items and creating Likert scales with only positive items (e.g., Cole et al, 2019;Ebesutani et al, 2012;Greenberger et al, 2003;Van Sonderen et al, 2013;Zhang et al, 2016). Several studies (e.g., Dueber et al, 2021;Greenberger et al, 2003;Zhang et al, 2016) have shown that when a balanced scale is changed to a scale with all PW items, the scale's factor structure becomes more consistent with the theoretical factor structure; in most cases, the scale becomes unidimensional. However, eliminating NW items cannot solve the other problems associated with the Likert format.…”

Who is flexible and adaptive in everyday life? Three facets of flexibility and development of the Flexibility Scale (FS-24)

“…The exceptions were two recoded, initially negatively-keyed items that showed loadings < .40 on the corresponding factors. Our results could be the sign of multidimensionality or other side effects of the negatively-keyed items, as discussed by Dueber et al (2021). In Study 2, the Flexibility Scale showed sufficient test-retest reliability after one week, suggesting the stability of flexibility as personality trait.…”

Working with AI: can stress bring happiness?

“…Employee engagement was measured based on Kumar and Pansari ( 2016 ). Self-esteem was assessed through 5 items adapted from the Rosenberg Self-Esteem Scale (SES) (Rosenberg 1965 , 1979 ), not regarding the reverse items as the literature suggests that negatively oriented items have a minor impact on instrument quality but influence measurement model and path coefficients (Dueber et al 2021 ). Diverse scales have been employed to assess happiness, such as the Subjective Happiness Scale (Nawijn and Peeters 2010 ) or the Satisfaction with Life Scale (SWLS) (Sirgy et al 2010 ; Nawijn 2011 ; Woo et al 2015 ), while another common approach (Van Boven and Gilovich 2003 ; Bimonte and Faralla 2015 ) involves a single item.…”