The Prediction of Training Proficiency in Firefighters: A Study of Predictive Validity in Spain

Berges, Alfredo; Río, Elena Fernández del; Ramos-Villagrasa, Pedro J.

doi:10.5093/jwop2018a2

Cited by 7 publications

(6 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In validity studies, the rating sources of job performance can be the supervisors (e.g., Harris et al, 1995; Campbell and Wiernik, 2015), instructors (e.g., Berges et al, 2018), peers (e.g., Harris and Schaubroeck, 1988; Viswesvaran et al, 2002), and incumbents (e.g., Bang and Reio, 2017; Jyoti and Sharma, 2017; Haider et al, 2018; Rehman and Shahnawaz, 2018). This paper focuses on the interrater reliability of supervisory performance ratings as they are the most frequently used in validity studies and for performance appraisal purposes (Bernardin and Beatty, 1984; Landy and Rastegary, 1989; Viswesvaran et al, 2002; Campbell and Wiernik, 2015).…”

Section: Introductionmentioning

confidence: 99%

Meta-Analysis of Interrater Reliability of Supervisory Performance Ratings: Effects of Appraisal Purpose, Scale Type, and Range Restriction

Salgado

Moscoso

2019

Front. Psychol.

View full text Add to dashboard Cite

Objectives: This reliability generalization study aimed to estimate the mean and variance of the interrater reliability coefficients (ryy) of supervisory ratings of overall, task, contextual, and positive job performance. The moderating effect of the appraisal purpose and the scale type was examined. It was hypothesized that the ratings collected for research purposes and multi-item scales have higher ryy. It was also examined whether ryy was similar for the four performance dimensions.Method: A database consisting of 224 independent samples was created and hierarchical sub-grouping meta-analyses were conducted.Results: The appraisal purpose was a moderator of ryy for the four performance dimensions. Scale type was a moderator of ryy for overall and task performance collected for research purposes. The findings also suggest that supervisors seem to have less difficulty evaluating overall job performance than task, contextual, and positive performance. The best estimates of the observed ryy for overall job performance are 0.61 for research-collected ratings and 0.45 for administrative-collected ratings.Conclusions: (1) Appraisal purpose moderates ryy and researchers and practitioners should be aware of its effects before collecting ratings or using empirically-derived interrater reliability distributions, (2) Scale type seems to moderate ryy in the case of the ratings collected for research purposes, only, (3) overall job performance is more reliably rated than task, contextual, and positive performance. Implications for research and practice are discussed.

show abstract

Section: Introductionmentioning

confidence: 99%

Meta-Analysis of Interrater Reliability of Supervisory Performance Ratings: Effects of Appraisal Purpose, Scale Type, and Range Restriction

Salgado

Moscoso

2019

Front. Psychol.

View full text Add to dashboard Cite

show abstract

“…General mental ability (GMA) tests are one of the most valid construct-based predictors of job performance and training success. Thousands of validity studies and many meta-analyses have shown that they are excellent predictors of different organizational criteria, such as supervisory ratings, work sample tests, job knowledge acquisition, grades, production records, instructor ratings, promotions, sales, and wages, and that the correlation between GMA and performance appears to be similar across jobs that differ considerably in content (Schmidt and Hunter, 1998; Murphy, 2002; Ones et al, 2012; Schmitt, 2014; Salgado, 2017a; Berges et al, 2018; Rodríguez and López-Basterra, 2018). Furthermore, the theoretical foundations of GMA are stronger than for any other personnel selection measure (Schmidt and Hunter, 1998; Hunt, 2011; Deary, 2012; Salgado, 2017a).…”

Section: Introductionmentioning

confidence: 99%

Meta-Analysis of the Validity of General Mental Ability for Five Performance Criteria: Hunter and Hunter (1984) Revisited

Salgado

Moscoso

2019

Front. Psychol.

View full text Add to dashboard Cite

This paper presents a series of meta-analyses of the validity of general mental ability (GMA) for predicting five occupational criteria, including supervisory ratings of job performance, production records, work sample tests, instructor ratings, and grades. The meta-analyses were conducted with a large database of 467 technical reports of the validity of the General Aptitude Test Battery (GATB) which included 630 independent samples. GMA showed to be a consistent predictor of the five criteria, but the magnitude of the operational validity was not the same across the five criteria. Results also showed that job complexity is a moderator of the GMA validity for the performance criteria. We also found that the GMA validity estimates are slightly smaller than the previous ones obtained by Hunter and Hunter (1984). Finally, we discuss the implications of these findings for the research and practice of personnel selection.

show abstract

“…This research has made several unique contributions to the literature on CR. The first unique contribution has been to show that CR is a relevant predictor of job performance and educational outcomes, and that the magnitude of the validity is similar or even more substantial than the validity of other well-studied predictors (e.g., cognitive abilities, personality dimensions; [88][89][90]). Moreover, in the case of academic performance, the findings showed that CR was a valid predictor for the three types of educational outcomes (i.e., GPA, UEM, and EG).…”

Section: Discussionmentioning

confidence: 99%

Cognitive Reflection and General Mental Ability as Predictors of Job Performance

2019

View full text Add to dashboard Cite

This paper presents a study with four independent samples on the validity of cognitive reflection (CR) for predicting job performance and academic outcomes. The results showed that CR was a valid predictor of academic outcomes (i.e., grades and exam marks) and job performance (i.e., assessment center ratings). The magnitude of the CR validity was similar to the validity of GMA. Moreover, we found that CR and GMA were moderately correlated, and multiple regression analyses showed that CR added a small amount of validity over GMA validity for predicting both types of performance. Finally, we discuss the implications of the findings for the theory and practice of predicting performance and organizational sustainability.

show abstract

The Prediction of Training Proficiency in Firefighters: A Study of Predictive Validity in Spain

Cited by 7 publications

References 23 publications

Meta-Analysis of Interrater Reliability of Supervisory Performance Ratings: Effects of Appraisal Purpose, Scale Type, and Range Restriction

Meta-Analysis of Interrater Reliability of Supervisory Performance Ratings: Effects of Appraisal Purpose, Scale Type, and Range Restriction

Meta-Analysis of the Validity of General Mental Ability for Five Performance Criteria: Hunter and Hunter (1984) Revisited

Cognitive Reflection and General Mental Ability as Predictors of Job Performance

Contact Info

Product

Resources

About