Item wording effects in self-report measures and reading achievement: Does removing careless respondents help?

The effects of reverse items on psychometric properties and respondents’ scale scores according to different item reversal strategies

“…Indeed, Schmitt and Allik (2005) and Hooper et al (2013) disclosed that reverse items were interpreted differently across countries. The mentioned contradiction may also be associated with the sample characteristics because the factors such as reading achievements (Bulut & Bulut, 2022;Michaelides, 2019), age levels (Bulut, 2021;Marsh 1986) and cognitive abilities (Gnambs & Schroeders, 2020) The results of the study indicated that internal consistency coefficient of Form-RN was lower than that of the other two versions. The facts that the reliability coefficient of Form-RN is lower than that of Form-P are coherent with those found by other researchers (Bourque & Shen, 2005;Carlson, 2011;Coleman, 2013;Ebesutani et al, 2012;Johnson et al, 2004;Roszkowski & Soven, 2010;Salazar, 2015;Suárez-Alvarez, 2018).…”

“…Also, Williams and Swanson (2001) (cited in Weems et al, 2006) found a similar relationship for adults. Likewise, Bulut and Bulut (2022) revealed that the severity of item wording effect that emerges because of reverse items is related to the reading ability. Hooper et al (2013) audited the behavior of reverse items on the Confidence in Mathematics Scale administered to students in TIMSS 2011.…”

“…A review of the literature disclosed that research about reverse items has a long history, and most studies focused on the first dilemma mentioned previously. The effects of reverse items on validity, internal consistency, factorial structure and factor loadings, item correlations, missing values, and respondents' ability estimation (mean scores on the scale) are principal subjects emphasized in these studies (e.g., Ahlawat, 1985;Bergstrom & Lunz, 1998;Boley et al, 2021;Bolt et al, 2020;Bulut & Bulut, 2022;Bulut, 2021;Chamberlain & Cummings, 1984;Conrad et al, 2004;Dooden, 2014;Dueber et al, 2021;Guyatt et al, 1999;Herche & England, 1996;Hooper et al, 2013, İlhan & Güler, 2017Kula Kartal, 2021;Kula Kartal et al, 2022;Locker et al, 2013;Marsh, 1996;Schriesheim & Hill, 1981;Salazar, 2015;Schotte et al, 1996;Suárez-Alvarez et al, 2018;Vigil-Colet et al, 2020). Conversely, the number of studies seeking answer to the second dilemma regarding reverse items is more limited.…”