The effect of alternative scoring methods on the analysis of rank order categorical data

Abstract

The purpose of this article is to examine the effects of five univariate scoring techniques for rank order categorical data and the results of analyses using each of the techniques for five-and ten-point bi-polar adjective scales. The effect of scoring method and scale length is assessed for the resultant distance to multivariate normality, inter-item reliability, discriminant analysis, least squares regression and logistic regression. For these data, the strongest effect of scoring was on distance to multivariate normality and determination of significant variables.

A comment on “using rank values as an interval scale” by dowling and midgley

“…This result also held for the certain subsets of the five-point data (c.f. Golden & Brockett, 1987). Ridit scoring, with one exception, was consistently closer to normality than any other scoring technique for both the five-point and ten-point scales and all groupings of the data on visit frequency.…”

“…They find that there is not substantive improvement obtained by the non-integer scoring method used, and call for more theoretical and empirical research on the topic. This comment extends their analysis by: (1) describing a general class of "distance" scoring methods which can be justified in an axiomatic fashion, (2) summarizing the empirical results obtained by Golden and Brockett (1987) comparing several different scoring methods with respect to several different statistical techniques, and (3) empirically assessing the effect of different scale lengths (e.g., a five point or a ten point scale) on the degree of approximation used in applying various interval level analysis to rank ordered categorical data. This comment addresses Dowling and Midgley's call for more research and finds the relative merits of non-integer scoring to be influenced by both scale length and type of analysis selected.…”

“…The interval treatment is probably the most pervasive approach to rank order categorical data and support for this stance can be found in Labovitz (1970), Morrison and Toy f1982), Wolf and Brockett (1979), and Dowling and Midgley (1991) where simulations demonstrate that using interval level formulae for ordinal data does not lead to large errors in particular instances. O'Brien (1979) and Golden and Brockett (1987) show, however, that the number of categories and the latent underlying distribution has an effect on the size of errors.…”

“…This section reviews the scoring systems to be investigated in this comment. Axiomatic justification for these scoring techniques can be found in Brockett (1981) and in Golden and Brockett (1987).…”

“…To investigate the effects of raw, ridit, normal mean, normal median and Blom scoring techniques for statistical analyses using both fiveand ten-point scales, a field study was designed and conducted. This section summarized results discussed in detail in Golden and Brockett (1987).…”