A Comparison of Holistic Versus Decomposed Rating of Position Analysis Questionnaire Work Dimensions

Abstract

Recent studies have attempted to reduce the cost and intrusiveness of the Position Analysis Questionnaire (PAQ) by limiting the amount of information provided to the analyst, with consistently negative results. We examined an alternative technique for improving the cost-effectiveness of the PAQ that avoids the need to rate the hundreds of items that constitute the instrument. Three groups of raters (professional job analysts, graduate students in industrial psychology who were familiar with the PAQ, and PAQ-unfamiliar undergraduates) made direct holistic ratings of the PAQ dimensions for four familiar jobs. The holistic ratings were compared with decomposed PAQ dimension profiles obtained from the item-level ratings of the professional analysts. Cronbach accuracy analyses indicated near-zero convergence between the holistic and decomposed dimension ratings, even for the professional PAQ job analysts. We conclude that holistic rating of dimensions is not an effective means of reducing the cost of a PAQ job analyses and that it is likely to be similarly ineffective with task-or ability-based instruments.Job analysis is ubiquitous in personnel psychology by virtue of the fact that its data form the foundation for virtually every aspect of human resource management. Unfortunately, most job analysis methods are time consuming and costly: considerable ratee and rater effort is spent observing jobs, providing background information, making ratings, and analyzing the results. Although precise figures are unavailable, it is likely that the high cost of the process deters many organizations from collecting and maintaining an adequate job analysis database. Consequently, it is not surprising that many recent job analysis studies (e.g.

Self-Presentation Processes in Job Analysis: A Field Experiment Investigating Inflation in Abilities, Tasks, and Competencies.

“…Similarly, Sackett, Cornelius, and Carron (1981) found no differences in job classification results when using a comprehensive task analysis or a simplified paired-comparison procedure. However, Butler and Harvey (1988) found virtually no convergence between holistic ratings of Position Analysis Questionnaire dimensions and decomposed ratings of individual Position Analysis Questionnaire items.…”

Holistic versus decomposed ratings of general dimensions of work activity

“…In these instances their holistic ratings might well diverge from ratings of each GWA produced via a detailed, decomposed-judgment process (Gibson et al, 2004). Additional studies demonstrated poor convergence between single-item ratings and decomposed scores in rating situations that were quite similar to the O*NET (for example, Butler and Harvey, 1988;DeNisi and Shaw, 1977;Harvey et al, 1994). Indeed, when the high cognitive demands inherent in the holistic judgment strategy are combined with the nebulous nature of the O*NET GWA ''operational definitions'', such divergence may be further exacerbated.…”

“…Based on prior research (Butler and Harvey, 1988;Harvey et al, 1994), we hypothesized that incumbents would produce inflated holistic ratings relative to analysts, and that low holistic-decomposed GWA convergence would be seen. The common-metric questionnaire (CMQ) was selected to provide the decomposed-judgment GWA scores in view of:…”

A perspective on work system analysis: classification and evaluation of methods

“…Evaluation of comprehensive methods relative to quality criteria: The PAQ (M cCormick et al 1969), AET (Rohmert and Landau 1983), and the Job Pro® le method (W agner 1985 ) represent the three most complete comprehensive methods based on extent of use, scope, and depth of analysis. Of these methods, the PAQ (M cCormick et al 1969) has been most often studied in terms of the quality criteria (M cCormick et al 1977, 1978, Colbert and Taylor 1978, Taylor and Colbert 1978, M cCormick 1979, Hoyos 1981, Shaw and Riskan d 1983, Cornelius et al 1984, Friedm an and Harvey 1986, Harvey and Hayes 1986, Carter and Biersner 1987, Butler and Harvey 1988, Sparrow 1989, M urphy 1991, M urphy et al 1991. The three comprehensive methods are evaluated according to criteria described by both Carm ines and Zeller (1979) and Petitti (1994) in table 5.…”