OBJECTIVE This study aimed to evaluate the effectiveness and efficiency of three short-listing methodologies for use in selecting trainees into postgraduate training in general practice in the UK.
METHODSThis was an exploratory study designed to compare three short-listing methodologies. Two methodologies -a clinical problem-solving test (CPST) and structured application form questions (AFQs) -were already in use for selection purposes. The third, a new situational judgement test (SJT), was evaluated alongside the live selection process. An evaluation was conducted on a sample of 463 applicants for training posts in UK general practice. Applicants completed all three assessments and attended a selection centre that used work-related simulations at final stage selection. Applicant scores on each short-listing methodology were compared with scores at the selection centre.RESULTS Results indicate the structured AFQs, CPST and SJT were all valid short-listing methodologies. The SJT was the most effective independent predictor. Both the structured AFQs and the SJT add incremental validity over the use of the CPST alone. Results show that optimum validity and efficiency is achieved using a combination of the CPST and SJT.CONCLUSIONS A combination of the CPST and SJT represents the most effective and efficient battery of instruments as, unlike AFQs, these tests are machine-marked. Importantly, this is the first study to evaluate a machinemarked SJT to assess non-clinical domains for postgraduate selection. Future research should explore links with work-based assessment once trainees are in post to address long-term predictive validity.selection
Medical Education 2011: 45: 289–297
Objectives This study aimed to examine candidate reactions to selection practices in postgraduate medical training using organisational justice theory.
Methods We carried out three independent cross‐sectional studies using samples from three consecutive annual recruitment rounds. Data were gathered from candidates applying for entry into UK general practice (GP) training during 2007, 2008 and 2009. Participants completed an evaluation questionnaire immediately after the short‐listing stage and after the selection centre (interview) stage. Participants were doctors applying for GP training in the UK. Main outcome measures were participants’ evaluations of the selection methods and perceptions of the overall fairness of each selection stage (short‐listing and selection centre).
Results A total of 23 855 evaluation questionnaires were completed (6893 in 2007, 10 497 in 2008 and 6465 in 2009). Absolute levels of perceptions of fairness of all the selection methods at both the short‐listing and selection centre stages were consistently high over the 3 years. Similarly, all selection methods were considered to be job‐related by candidates. However, in general, candidates considered the selection centre stage to be significantly fairer than the short‐listing stage. Of all the selection methods, the simulated patient consultation completed at the selection centre stage was rated as the most job‐relevant.
Conclusions This is the first study to use a model of organisational justice theory to evaluate candidate reactions during selection into postgraduate specialty training. The high‐fidelity selection methods are consistently viewed as more job‐relevant and fairer by candidates. This has important implications for the design of recruitment systems for all specialties and, potentially, for medical school admissions. Using this approach, recruiters can systematically compare perceptions of the fairness and job relevance of various selection methods.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.