Assessor Constructs in Use as the Missing Component in Validation of Assessment Center Dimensions: A critique and directions for research

We clarify the existing literature on gender differences in work sample exams by disentangling such differences on work sample exams from other predictors (e.g., situational judgment tests). In fact, we note that there are only two articles specifically related to gender differences in work sample exams. Based on theory and literature from social psychology, neuropsychology, and applied psychology, we propose that the three constructs of selfconcept, social skills, and writing skills are likely to influence work sample gender differences. We tested our hypotheses on two samples of managers. We found in one instance that males scored higher on a technical exercise, but there was stronger support for females, on average, scoring higher on exercises that involved social skills and on exercises that involved writing skills. Work samples that targeted a broad array of knowledge, skills, and abilities were associated with higher overall scores for females (ds of À.37 and À.34) and, thus, were unlikely to be associated with gender-based adverse impact against females.

“…We also note the intercorrelations among our assessments of the key skills (as per Jones & Born, 2008). We generally found low intercorrelations with most values between about .1 and .3.…”

Section: Written Skills Differencesmentioning

confidence: 63%

Section: Job 2 -Resultsmentioning

confidence: 85%

See 1 more Smart Citation

Work Sample Exams and Gender Adverse Impact Potential: The influence of self‐concept, social skills, and written skills

Roth

Buster²,

Barnes-Farrell

2010

“…A useful approach to resolve this issue is to group similar manifest dimensions into broader dimension factors represented at the latent level (Hoffman, Melchers, Blair, Kleinmann, & Ladd, ). Indeed, similar manifest dimensions have been grouped into broader underlying dimension factors from the inception of the AC method (Bray, ; OSS Assessment Staff, ), a practice that persists today in theoretical (Jones & Born, ), conceptual (Arthur et al., ), empirical (Kolk, Born, & van der Flier, ; Meriac, Hoffman, Woehr, & Fleisher, ; Schmitt, ; Shore, Thornton, & Shore, ), and practical (Howard, ) applications of the AC method. Thus, although past research has sporadically sought to ensure the psychometric soundness of espoused constructs, the more common approach is to simply take espoused dimensions at face value.…”

Section: Introductionmentioning

confidence: 99%

Alternate Approaches to Understanding the Psychometric Properties of Assessment Centers: An analysis of the structure and equivalence of exercise ratings

Hoffman

Meade

2012

The present study uses an alternate analytical framework to examine the degree to which performance is differentiated by dimensions in assessment center (AC) exercises and whether these performance dimensions are rated on the same scale across exercises. Confirmatory factor analysis likelihood ratio tests supported the presence of three broad latent performance dimensions in each of three AC exercises. Additional tests revealed that five of six manifest performance dimensions were rated on the same psychological scale across exercises. Taken together, our results support a multidimensional interpretation of AC exercises and provide empirical support to the notion that differences in AC performance across exercises reflect true performance, rather than a measurement artifact.

“…AC designers and assessors are required to hold expertise relevant to their positions. The views of subject matter experts (SMEs) are widely used as a source of research evidence in applied psychology (Dries, Pepermans, & Carlier, 2008;Flores, Zheng, Rau, & Thomas, 2012;Francis-Smythe, Haase, Thomas, & Steele, 2013;Jones & Born, 2008;Motowidlo & Beier, 2010;Sebastian, Ramos, Stumbo, McGrath, & Fairbrother, 2014;Woehrmann, Deller, & Wang, 2014). Although AC assessees are not organizational SMEs in the usual sense of the term, they have relevant experience of ACs from a distinctive perspective, and their perceptions may impact in several ways on organizations using ACs, including the likelihood that they will accept a job offer and their first impression of the organization.…”

Section: Introductionmentioning

confidence: 99%

The Perceived Nature and Incidence of Dysfunctional Assessment Center Features and Processes

Dewberry

Jackson

2016

The aims of this research are to identify, for the first time, the dysfunctional features and processes perceived to take place in assessment centers (ACs) from multiple perspectives (assessment center designers, assessors, and candidates) and to indicate the frequency of these phenomena. Two surveys were conducted in this study. In the first, a wide variety of dysfunctional processes and events were identified, and, in the second, many of these processes are reported to occur with regularity. Based on these findings, it is proposed that ACs should be construed, researched, and managed, not only as large-scale psychometric systems, but also as complex administrative, social, and political events susceptible to a broad range of dysfunctional phenomena.