Seeking a better balance between efficiency and interpretability: Comparing the likert response format with the Guttman response format.

Abstract

The Likert item response format for items is almost ubiquitous in the social sciences and has particular virtues regarding the relative simplicity of item-generation and the efficiency for coding responses. However, in this article, we critique this very common item format, focusing on its affordance for interpretation in terms of internal structure validity evidence. We suggest an alternative, the Guttman response format, which we see as providing a better approach for gathering and interpreting internal structure validity evidence. Using a specific survey-based example, we illustrate how items in this alternative format can be developed, exemplify how such items operate, and explore some comparisons between the results from using the two formats. In conclusion, we recommend usage of the Guttman response format for improving the interpretability of the resulting outcomes. Finally, we also note how this approach may be used in tandem with items that use the Likert response format to help balance efficiency with interpretability.

“Anyone Can Be a Researcher”: Findings from the MYHealth Virtual Summer Research Training Program for High School Students from Historically Marginalized Groups

“…Researcher Identity. The Researcher Identity Scale measures students' agency, community, interest, aspiration, and self-perceptions as someone who can contribute to investigations about a topic (Koo et al, 2021), and has been validated for use with high school students as a Likert-scale and Guttman scale (Wilson et al, 2022). In this study, researcher identity was derived by averaging the Guttman scale answers to twelve questions (Wilson et al, 2022).…”

“…The Researcher Identity Scale measures students' agency, community, interest, aspiration, and self-perceptions as someone who can contribute to investigations about a topic (Koo et al, 2021), and has been validated for use with high school students as a Likert-scale and Guttman scale (Wilson et al, 2022). In this study, researcher identity was derived by averaging the Guttman scale answers to twelve questions (Wilson et al, 2022). When answering each question, respondents could choose from five text options, which translated to a numeric range from 1 to 5, where the first option (assigned the numerical value of 1 for analysis purposes) was the "lowest level" and the last option (assigned the numerical value of 5 for analysis purposes) was the "highest level".…”

“…Data Collection. At baseline and upon completion of the program, participants completed an online survey which included items on science interest, science motivation, and researcher identity (Aschbacher et al, 2009;Glynn et al, 2011;Wilson et al, 2022). Demographic information was also collected in the baseline survey.…”

Assessing Self-Efficacy Growth in Innovation Skills Using a Developmental Perspective

“…Though there is widespread use of these procedures in survey research, literature within the field of measurement has cautioned the inherent assumptions tied to these analyses (Boone, 2016;Wilson et al, 2022). Harwell and Gatti (2001) classified such ordinal-to-interval conversion procedures as an appeal to Classical Test Theory (CTT), a measurement method with widely recognized limitations as compared to more robust psychometric methods like Item Response Theory (IRT; Wang, 1999).…”

An Overview of Alternative Formats to the Likert Format: A Comment on Wilson et al. (2022).

“…

Wilson et al (2022) compared the Likert response format to an alternative format, which they called the Guttman response format. Using a Rasch modelling approach, they found that the Guttman response format had better properties relative to the Likert response format.

…”