An Examination of the Validity and Incremental Value of Needed-at-Entry Ratings for a Customer Service Job

Abstract

We examined needed-at-entry ratings of knowledge, skills, abilities, and other characteristics (KSAOs) provided by job experts (N = 285) during a job analysis for a customer service manager position. To serve as an external referent, a group of industrial and organisational psychologists (N = 31) rated the perceived trainability of each KSAO. Analyses revealed only limited support for the validity of inferences drawn from job experts' needed-at-entry ratings. Consistent with our hypotheses, less validity evidence was found for ratings of the more person-oriented and abstract "AO" attributes than for the more job-oriented and concrete "KS" attributes. In general, job experts tended to rate attributes as needed-at-entry that psychologists thought could be developed on the job. We also found that failing to collect needed-at-entry ratings, and relying on only the more common importance-to-the-job ratings, would have resulted in a rather different set of critical KSAOs identified for assessment during the selection process.

Job analysis ratings and criterion‐related validity: Are they related and can validity be used as a measure of accuracy?

“…Past research on measuring job analysis accuracy as outcomes As alternatives to reliability-based measures of accuracy, it has been suggested that researchers shift focus to the consequences associated with the job analysis (Sanchez & Levine, 2000) or the validity of inferences drawn from the job analysis (Morgeson & Campion, 2000). Despite having been suggested more than 18 years ago, relatively little research has been conducted examining the consequences of job analysis (c.f., Jones et al, 2001;Levine, Ash, & Bennett, 1980;Manson, 2004;Van Iddekinge, Raymark, & Eidson, 2011). While research is only beginning to shift from reliability to consequences, examining job analysis methods and results in the light of the purpose they were collected to serve offers an important new lens for research.…”

Work analysis for assessment.

“…For example, Jones et al (2001) found that job analysts made better predictions of worker attribute trainability than incumbents and students when trainability ratings were compared with actual changes in pre-and posttest learning measures. Similarly, Van Iddekinge, Raymark, and Edison (2011) correlated ratings of the extent to which KSAOs were needed at entry with ratings of perceived KSAO trainability made by a panel of psychologists. Their findings indicated that ratings of the more abstract ability and other characteristic attributes were less valid than those of more concrete knowledge and skill attributes.…”