Are assessment center behaviors' meanings consistent across exercises? A measurement invariance approach

Abstract

To examine the appropriateness of a Multi‐Trait–Multi‐Method framework for testing construct validity of Assessment Centers (ACs) and get practical implications for the improved AC design, degree to which the AC dimension‐related performance behaviors consistently manifest across multiple AC rating situations was investigated. The present study used a large sample (N = 5,006) to apply a measurement invariance analysis. AC rating situations generally produced consistent factor loadings for items on AC dimensions, item residuals, dimension factor variances, and covariance between dimensions. The AC rating situation of interview tended to produce higher ratings and less item residuals. These findings support the consistency in constructs assessed across different AC rating situations, while some exercises may be better for teasing apart particular dimensions than others.

A Review of Measurement Equivalence in Organizational Research: What's Old, What's New, What's Next?

“…Finally, when assessments involve ratings from other people, rater effects can also cause MNE. As a result, testing the equivalence of ratings across assessment methods (e.g., Lee et al 2017) and raters (e.g., Bynum et al 2013) is important to ensure that individuals are not being evaluated on different criteria.…”