Moderated regression analysis and Likert scales: Too coarse for comfort.

Abstract

One of the most commonly accepted models of relationships among three variables in applied industrial and organizational psychology is the simple moderator effect. However, many authors have expressed concern over the general lack of empirical support for interaction effects reported in the literature. We demonstrate in the current sample that use of a continuous, dependent-response scale instead of a discrete, Likert-type scale, causes moderated regression analysis effect sizes to increase an average of 93%. We suggest that use of relatively coarse Likert scales to measure fine dependent responses causes information loss that, although varying widely across subjects, greatly reduces the probability of detecting true interaction effects. Specific recommendations for alternate research strategies are made.

Inside opportunity formation: enterprise failure, cognition, and the creation of opportunities

“…Providing support for H2c and H2e, recognition of failure was found to signifi cantly interact within the relationship between start-up decision-making expertise and perceived chance of new venture success (β = 0.17, p < 0.05) and between start-up experience and perceived chance of new venture success (β = −0.16, p < 0.05). While the effect size 2 of 0.02 is low (Cohen, 1988), it is not disconfi rming, since a small effect size does not necessarily mean an unimportant effect-particularly because strong, unambiguous results in support of interaction effects tend to be rare (Russell and Bobko, 1992). In this way even small interaction effects can be meaningful (Chin et al, 2003).…”

In Pursuit of Moderation: Nine Common Errors and Their Solutions

“…Subjects faced with reporting Y responses on a five-point Likert scale must somehow reduce their latent 7-to 25-point dependent Y response into the relatively coarse five-point overt response format. Russell and Bobko (1992) found subjects in this exact scenario using the model Y = X'Z and facing a 150-point overt response scale yielded a ARM MR effect size that was 97 percent larger than subjects faced with placing overt Y responses on a traditional five-point Likert scale. It is important to place this result in context of Likert's (1932) oft replicated finding that increasing the number of response categories beyond five to seven does not yield substantial gains in observed reliability (cf.…”

Ten Common Misunderstandings, Misconceptions, Persistent Myths and Urban Legends about Likert Scales and Likert Response Formats and their Antidotes

“…Scaling the data so it is "perfectly interval data," it should be noted, will improve (in most cases) the Pearson correlation coefficients, which in turn will have very significant multiplier effects of various kinds in all statistical analyses that use the correlation coefficient as one of if not the fundamental unit of input to the analysis (e.g., multiple regressions, factor and discriminant analyses, and the multivariate F-test). So it is really the correlation coefficient that is most effected by "scale" and "data" type, which is the real, core and key problem that is never mentioned or discussed by various experts on Likert scale, Likert response formats, and statistical analyses thereof, with one notable excellent exception [32] . F is not made of glass but correlation coefficients are to a great degree, and this particular empirical fact and its many consequences are one of the greatest silences in all of this literature.…”