Accuracy or consequential validity: which is the better standard for job analysis data?

Abstract

The value of research on the accuracy of job analysis is questioned. It is argued that the traditional criteria employed to evaluate job analysis accuracy (i.e., interrater agreement and deviations from proxy true scores) provide information of little practical value. Alternative criteria focusing on the consequences of job analysis data are suggested. Consequence-oriented criteria are clari®ed through a review of the various inferential leaps or decision points that job analysis supports. In addition, the consequences of job analysis are also thought to be a function of the rules governing the making of jobanalysis-based inferences which, unfortunately, are sometimes unspeci®ed in even the most molecular job analysis methodologies.

Lack of consensus among competency ratings of the same occupation: Noise or substance?

“…This study indicated that up to 25% of the variance in competency ratings formulated by incumbents of the same occupation was related to these occupational factors. Thus, our results support the notion that rating differences among incumbents of the same occupation capture not solely random error variance, but also substantive factors (Dierdorff & Morgeson, 2007;Sanche z et al, 1998;Sanchez & Levine, 2000).…”

“…Thus, rather than assuming that disagreement is a definite sign of rating inaccuracy, our study builds on an emerging stream of work analytic research that illustrates how further considering the roots of such disagreement provides fruitful information regarding the manner in which incumbents perceive and construe their jobs (Di erdorff & Morgeson, 2007;Sanchez et al, 1997Sanchez et al, , 1998Sanchez & Levine, 2000).…”

“…Other theories, including role theory, share this view of incumbents as active agents who customize and/or change their job to fit their role identity, past experience, motivation, and personal and professional goals (see also Dierdorff, Rubin, & Morgeson, 2009;Grant, 2007;Jackson, 1981;Roberts et al, 2005). The key implication for our study is that, under this new prism, differences among incumbents of the same job, which traditional job analysis deemed to be the by-product of biases and carelessness, may instead capture legitimate differences in the unique way in which incumbents define and shape their job (Sanchez & Levine, 2000).…”

Effects of job satisfaction, affective commitment and job involvement on job skill ratings

“…In HR practices (such as recruitment, training or appraisal, etc. ), inaccurate job analysis affects other HR activities based on it (Sanchez and Levine, 2000). Studies focusing on factors influencing job analysis ratings are, therefore, of basic significance to HR management practices.…”

Assessing Training Needs: Do Work Experience and Capability Matter?

“…This assumes, of course, that the TNA ratings pro-RATINGS OF TRAINING NEEDS vided by job incumbents are indeed valid or accurate. Important to note is that the validity and accuracy of work requirement ratings can be viewed from several perspectives, ranging from the quality of inferences drawn from the data (Morgeson & Campion, 2000) to the eventual use of the data ("consequential validity"; Sanchez & Levine, 2000). Our study did not seek to examine the validity or accuracy of TNA ratings but rather investigated individual-level variables to test whether they could account for variance in task-and skill-related TNA ratings.…”