Comparison of Four Methods for Weighting Multiple Predictors

Abstract

An important issue in personnel psychology concerns the methods that are used to combine multiple criteria and/or multiple predictors into a single composite. Fralicx and Raju (1982) looked at five methods of combining multiple criteria and concluded that with the exception of canonical weights, the results obtained from four of the methods were almost identical. The present study differed from Fralicx and Raju (1982) in that it used predictors rather than criteria, was composed of a different type of sample, used different types of weighting methods, a smaller number of dimensions, and included a cross-validation procedure. In spite of these differences the conclusion drawn in both studies is similar. That is, most weighting methods are highly related. Furthermore, the present study demonstrated that no weighting method was superior to another in terms of protection from validity shrinkage.

The Usefulness of Unit Weights in Creating Composite Scores

“…1 Highly similar results have also been noted by other researchers (Grant & Bray, 1970;Lawshe, 1969;Trattner, 1963;Wallace & Schwab, 1976). The recent literature provides similar findings-that is, very high correlations between unit-weighted and regression composite scores (Aamodt & Kimbrough, 1985;Fralicz & Raju, 1982;Ree, Carretta, & Earles, 1998). 2 For example, Fralicz and Raju (1982) found that expert-generated weights (i.e., by a subject matter expert), equal raw score weights, and unit weights of a four-test composite predicted performance for bank tellers.…”

“…Unit weights are robust to outliers in the data because, as noted earlier, the weights are set a priori (e.g., Aamodt & Kimbrough, 1985). In contrast, the estimation of regression weights is susceptible to outliers/influential cases (for a review, see Roth & Switzer, 2002).…”

Multiple test batteries as predictors for pilot performance: A meta‐analytic investigation

“…Even with the extensive use of TBs in selection, the existing meta‐analyses of criterion‐related validity have focused primarily on specific ability test scores and paid less attention to the composite scores derived from multiple ability TBs. An understanding of the composite scores produced by a certain battery is critical for interpreting its psychometric properties (e.g., Aamodt & Kimbrough, ; Bobko, Roth, & Buster, ). Hence, to clearly understand the validity of composite scores in the selection context, it is important to investigate the ability domain that influences and explains the derived scores.…”

A More Consistent Application of the RNR Model

“…Although weighting techniques and subgroup norming procedures have been utilized and discussed for many years in other fields (Aamodt & Kimbrough, 1985; Wang & Stanley, 1970) and have recently been found to increase accuracy for offender models (Barnoski & Aos, 2003; Duwe, 2014; Hamilton et al, 2016), these techniques have been largely overlooked when assessing offender needs. Thus, the development methods created for the STRONG-R risk assessment were used (see Hamilton et al, 2016) to further provide methodological advancements for items selection and weighting that have been found to increase predictive performance over models known to use bivariate selection and Burgess (unweighted) methodologies.…”