Criterion validation of two approaches to performance appraisal: The behavioral observation scale and the relative percentile method

Goffin, Richard D.; Gellatly, Ian R.; Paunonen, Sampo V.; Jackson, Douglas N.; Meyer, John P.

doi:10.1007/bf02278252

Cited by 71 publications

(97 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Evidence shows that the RPM can achieve a greater range of-and more accuratescores when compared with rating each student independently. 9 The three domain ratings were strongly correlated with one another (r = 0.71-0.84, P < .001). A principal components factor analysis of the three domain ratings showed that a single factor could explain 85% of the variance.…”

Section: In-role Performancementioning

confidence: 89%

The Predictive Validity of a Text-Based Situational Judgment Test in Undergraduate Medical and Dental School Admissions

et al. 2017

View full text Add to dashboard Cite

Problem Situational judgment tests (SJTs) can be used to assess the nonacademic attributes necessary for medical and dental trainees to become successful practitioners. Evidence for SJTs’ predictive validity, however, relates predominantly to selection in postgraduate settings or using video-based SJTs at the undergraduate level; it may not be directly transferable to text-based SJTs in undergraduate medical and dental school selection. This preliminary study aimed to address these gaps by assessing the validity of the UK Clinical Aptitude Test (UKCAT) text-based SJT. Approach Study participants were 218 first-year medical and dental students from four UK undergraduate schools who completed the first UKCAT text-based SJT in 2013. Outcome measures were educational supervisor ratings of in-role performance in problem-based learning tutorial sessions—mean rating across the three domains measured by the SJT (integrity, perspective taking, and team involvement) and an overall judgment of performance—collected in 2015. Outcomes There were significant correlations between SJT scores and both mean supervisor ratings (uncorrected r = 0.24, P < .001; corrected r = 0.34) and overall judgments (uncorrected r s = 0.16, P < .05; corrected r s = 0.20). SJT scores predicted 6% of variance in mean supervisor ratings across the three nonacademic domains. Next Steps The results provide evidence that a well-designed text-based SJT can be appropriately integrated, and add value to, the selection process for undergraduate medical and dental school. More evidence is needed regarding the longitudinal predictive validity of SJTs throughout medical and dental training pathways, with appropriate outcome criteria.

show abstract

Section: In-role Performancementioning

confidence: 89%

The Predictive Validity of a Text-Based Situational Judgment Test in Undergraduate Medical and Dental School Admissions

et al. 2017

View full text Add to dashboard Cite

show abstract

“…** Figure 1 about here** As is customary in meta-analysis (e.g., Dudley et al, 2006;Salgado, 1998), composite correlation coefficients were obtained whenever more than one predictor measure had been used to assess the same constructs; such as in a study by Marcus, Goffin, Johnston and Rothstein (2007), where two different personality measures were employed to assess the FFM dimensions. Composite coefficients were also calculated where more than one criterion measure had been used; such as in research comparing the usefulness and psychometric properties of two methods of performance appraisal aimed at measuring the same criterion scales (Goffin, Gellatly, Paunonen, Jackson & Meyer, 1996).…”

Section: The Choice Of Predictor and Criterion Variablesmentioning

confidence: 99%

The Relationships Between Traditional Selection Assessments and Workplace Performance Criteria Specificity: A Comparative Meta-Analysis

2015

View full text Add to dashboard Cite

. (2015 The relationship between traditional selection assessments and workplace performance criteria specificity: A comparative metaanalysis. Human Performance 28.1, 1-25 AbstractIndividual workplace performance is a crucial construct in Work Psychology. However, understanding of its conceptualization remains limited, particularly regarding predictor-criterion linkages. This study examines to what extent operational validities differ when criteria are measured as overall job performance compared to specific dimensions as predicted by ability and personality measures respectively. Building on Bartram's work (2005), systematic review methodology is used to select studies for meta-analytic examination. We find validities for both traditional predictor types to be enhanced substantially when performance is assessed specifically rather than generically. Findings indicate assessment decisions can be facilitated through a thorough mapping and subsequent use of predictor measures using specific performance criteria. We discuss further implications, referring particularly to the development and operationalization of even more finely grained performance conceptualizations.

show abstract

“…The Relative Percentile Method (RPM) formed the basis for the rating scale used to assess team learning, which provided a meaningful comparison point for behaviors occurring in previous weeks 25 . More specifically, teams responded collectively to two RPM items measuring: (1) exploratory and (2) exploitative learning activities.…”

Section: Methods and Proceduresmentioning

confidence: 99%

“…In the aforementioned TA training, we included coaching to calibrate TAs perceptions of what constitutes innovation before they completed their ratings (see appendix B). The RPM formed the basis for the rating scale used to measure innovation 25 . Team innovation was defined as the extent to which the team's prototype embodied both a) the existence of a novel, unique, and original idea, and b) the effective implementation of the idea and functionality of the prototype 21 .…”

Section: Methods and Proceduresmentioning

confidence: 99%

Team Learning Behaviors: Supporting Team-Based Learning in a First-Year Design and Communications Course

Larson¹,

Hoffart²,

O'Neill³

et al.

2016 ASEE Annual Conference &Amp; Exposition Proceedings

View full text Add to dashboard Cite

Nicole is completing her PhD in Industrial Organizational Psychology at the University of Calgary under the supervision of Dr. Thomas O'Neill. She has been working with the Schulich School of Engineering for the past three years. During this period she has been involved in several initiatives, such as assessing student learning and engagement, implementing systems for peer evaluations, and leading teamwork training sessions. Nicole is currently conducting research on team learning processes in engineering student project teams. Additionally, she has co-developed a framework for measuring and interpreting an array of team dynamics. An online assessment tool has been created based on this framework which allows teams to diagnose and improve the "health" of their team. She is passionate about her area of research and plans to continue conducting research on factors that contribute to effective teamwork. Ms. Genevieve Hoffart, University of CalgaryGenevieve is a first year M.Sc. student under the supervision of Dr. Thomas O'Neill at the University of Calgary focusing on at team dynamics, training, and communication. She has been working with the Schulich School of Engineering for the past four years during which time her focus has been on improving team dynamics and maximizing the student experience. In addition co-developing a communication training framework that has now been applied to over 3500 students campus wide, Genevieve has personally facilitated many of the training sessions. Her goal is to continue working on developing applicable and universal tools to improve the experience and functioning of student teams in institutions across North America. Dr. Tom O'Neill, University of CalgaryO'Neill is a Professor of Industrial/Organizational Psychology and a leading expert in the areas of team dynamics, virtual teams, conflict management, personality, and assessment. He is director of the Individual and Team Performance Lab and the Virtual Team Performance, Innovation, and Collaboration Lab at the University of Calgary, which was built through a $500K Canada Foundation for Innovation Infrastructure Grant. He also holds operating grants of over $300K to conduct leading-edge research on virtual team effectiveness. Over the past 10 years, Tom has worked with organizations in numerous industries, including oil and gas, healthcare, technology, and venture capitals. He is currently engaged with the Schulich School of Engineering at the University of Calgary to train, develop, and cultivate soft-skill teamwork competencies in order to equip graduates with strong interpersonal and communication capabilities. Prof. Marjan Eggermont, University of CalgaryMarjan Eggermont is the current Associate Dean (Student Affairs) and a Senior Instructor and faculty member at the University of Calgary in the Mechanical and Manufacturing department of the Schulich School of Engineering, University of Calgary. She teaches graphical, written and oral communication in their first Engineering Design and Communication course taught to all i...

show abstract

Criterion validation of two approaches to performance appraisal: The behavioral observation scale and the relative percentile method

Cited by 71 publications

References 9 publications

The Predictive Validity of a Text-Based Situational Judgment Test in Undergraduate Medical and Dental School Admissions

The Predictive Validity of a Text-Based Situational Judgment Test in Undergraduate Medical and Dental School Admissions

The Relationships Between Traditional Selection Assessments and Workplace Performance Criteria Specificity: A Comparative Meta-Analysis

Team Learning Behaviors: Supporting Team-Based Learning in a First-Year Design and Communications Course

Contact Info

Product

Resources

About