A Continuous Solution to the Norming Problem

Abstract

Conventional methods for producing test norms are often plagued with “jumps” or “gaps” (i.e., discontinuities) in norm tables and low confidence for assessing extreme scores. We propose a new approach for producing continuous test norms to address these problems that also has the added advantage of not requiring assumptions about the distribution of the raw data: Norm values are established from raw data by modeling the latter ones as a function of both percentile scores and an explanatory variable (e.g., age). The proposed method appears to minimize bias arising from sampling and measurement error, while handling marked deviations from normality—such as are commonplace in clinical samples. In addition to step-by-step instructions in how to apply this method, we demonstrate its advantages over conventional discrete norming procedures using norming data from two different psychometric tests, employing either age norms ( N = 3,555) or grade norms ( N = 1,400).

Bias-variance trade-off in continuous test norming

“…Continuous norming (e.g., Zachary & Gorsuch, 1985;Lenhard, Lenhard, Suggate, & Segerer, 2018;Oosterhuis, 2017) explicitly builds upon the assumption that test score distributions change smoothly with age. It is thus more appropriate than traditional norming for those psychological tests that show this gradual change in test performance with age.…”

“…For those conditional distributions that are non-normal and/or heteroscedastic, more general regression models can be used to estimate normed scores. This can be done in nonparametric (Lenhard et al, 2018) and parametric ways.…”

“…In continuous norming practice, the relationship between the mean test score and the predictor is sometimes assumed to be linear (e.g., Ganguli et al, 2010;Grober et al, 2015;Agelink van Rentergem, de Vent, Schmand, Murre, & Huizinga, 2018). Nonlinear, yet smooth relationships are modelled by including a second order polynomial of the predictor (e.g., Van der Elst, Hoogenhout, Dixon, De Groot, & Jolles, 2011;Goretti et al, 2014;Kirsebom et al, 2019) or higher-order polynomials (e.g., Lenhard et al, 2018), or by using splines (e.g., Rommelse et al, 2018). Homoscedasticity and normality of the conditional score distribution are often assumed in norming practice (e.g., Van Breukelen & Vlaeyen, 2005;Goretti et al, 2014;Grober et al, 2015).…”

“…Fourth, we only explored estimation through GAMLSS to deal with violated assumptions of the standard regression model. Alternatives are semi-parametric continuous norming (Lenhard et al, 2018) and robust regression (Wilcox, 2012). The semi-parametric approach performed similar to the use of specific GAMLSS distributions, except for extreme (i.e., easy and difficult) tests for the population of interest, where the semi-parametric approach outperformed (Lenhard, Lenhard, & Gary, 2019).…”

Regression-based normative data for social health scale for the elderly (short version) in eastern China

“…However, this norm has not been validated in other populations in Zhejiang Province yet. Recently, an alternative approach to derive continuous norms using multivariate regression models have been proposed [11] and widely used in neuropsychological and social related measures [12][13][14]. This regression-based approach is useful insofar as it allows an individual's predicted score on a measure to reflect specific demographic characteristics [15].…”