Components of postgraduate competence: analyses of thirty years of longitudinal data

Hojat, Mohammadreza; Paskin, David; Callahan, Clara A.; Nasca, Thomas J.; Louis, Daniel Z.; Veloski, J. Jon; Erdmann, James B.; Gonnella, Joseph S.

doi:10.1111/j.1365-2923.2007.02841.x

Cited by 32 publications

(23 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Several groups have assessed the predictive validity of other high‐stakes medical examinations, including the United States Medical Licensing Examination ® (USMLE ® )8, 9, 10, the Quebec Licensing Examination11, the Medical Council of Canada Qualifying Examination12, 13, 14, the MRCP examination15, 16, 17 and Membership of the Royal College of General Practitioners examination15. Performance in these examinations predicts performance in clinical practice and other medical assessment processes11 18, 19.…”

Section: Discussionmentioning

confidence: 99%

Impact of performance in a mandatory postgraduate surgical examination on selection into specialty training

et al. 2017

View full text Add to dashboard Cite

BackgroundThe Intercollegiate Membership of the Royal College of Surgeons (MRCS) examination is undertaken by large numbers of trainees in the UK and internationally as a mandatory step within surgical training. Unlike some high‐stakes medical examinations, the MRCS is yet to be validated. A quantitative study was undertaken to assess its predictive validity by investigating the relationship between MRCS (Parts A and B) and national selection interview scores for general and vascular surgery in the UK.MethodsPearson correlation coefficients were used to examine the linear relationship between each assessment, and linear regression analyses were employed to identify potential independent predictors of the national selection score. All UK medical graduates who attempted the interview in 2011–2015 were included.ResultsSome 84·4 per cent of the candidates (1231 of 1458) were matched with MRCS data. There was a significant positive correlation between the first attempt score at Part B of the MRCS examination and the national selection score (r = 0·38, P < 0·001). In multivariable analysis, 17 per cent of variance in the national selection first attempt score was explained by the Part B MRCS score and number of attempts (change in R 2 value of 0·10 and 0·07 respectively; P < 0·001). Candidates who required more than two attempts at Part B were predicted to score 8·1 per cent less than equally matched candidates who passed at their first attempt.ConclusionThis study supports validity of the MRCS examination, and indicates its predictive value regarding entry into specialist training.

show abstract

Section: Discussionmentioning

confidence: 99%

Impact of performance in a mandatory postgraduate surgical examination on selection into specialty training

et al. 2017

View full text Add to dashboard Cite

show abstract

“…Based on prior research we hypothesized two equally likely factor analytic solutions. First, research exploring the dimensions of clinical performance of medical students and residents has consistently demonstrated two main factors or themes, namely a cognitive (knowledge/technical) dimension and an affective (interpersonal/professionalism) dimension (Wilkinson and Frampton 2003;Hojat et al 2007;Greenburg et al 2007;Nasca et al 2002;Silber et al 2004;Thomas et al 1999;Volkan et al 2004). Alternatively, the research cited above suggests a single dimension for mini-CEX scores.…”

Section: Introductionmentioning

confidence: 94%

Internal structure of mini-CEX scores for internal medicine residents: factor analysis and generalizability

Cook

Beckman

Mandrekar

et al. 2010

Adv in Health Sci Educ

View full text Add to dashboard Cite

the mini-CEX appears to measure a single global dimension of clinical competence. If educators desire to measure discrete clinical skills, alternative assessment methods may be required. Our approach to factor analysis overcomes the limitation of repeated observations on subjects without discarding data, and may be useful to other researchers attempting factor analysis of datasets in which individuals contribute multiple observations.

show abstract

“…Outcome evaluation has only been attempted in a few universities such as Thomas Jefferson Medical College, in which the clinical competence of 4,560 graduates between 1975 and 2004 were rated by the program directors of their hospitals (Hojat et al 2007). Apart from longitudinal analyses, ratings by program directors or other staff members have been investigated for reliability and validity in assessing pediatric trainees' clinical performance (Archer et al 2010) and physicians' professionalism (Cruess et al 2006;Tsugawa et al 2011).…”

Section: Introductionmentioning

confidence: 99%

Development of a Peer Review System Using Patient Records for Outcome Evaluation of Medical Education: Reliability Analysis

Kameoka

Okubo

Koguma

et al. 2014

Tohoku J. Exp. Med.

View full text Add to dashboard Cite

In addition to input evaluation (education delivered at school) and output evaluation (students' capability at graduation), the methods for outcome evaluation (performance after graduation) of medical education need to be established. One approach is a review of medical records, which, however, has been met with difficulties because of poor inter-rater reliability. Here, we attempted to develop a peer review system of medical records with high inter-rater reliability. We randomly selected 112 patients (and finally selected 110 after removing two ineligible patients) who visited (and were hospitalized in) one of the four general hospitals in the Tohoku region of Japan between 2008 and 2012. Four reviewers, who were well-trained general internists from outside the Tohoku region, visited the hospitals independently and evaluated outpatient medical records based on an evaluation sheet that consisted of 14 items (3-point scale) for record keeping and 15 items (5-point scale) for quality of care. The mean total score was 84.1 ± 7.7. Cronbach's alpha for these items was 0.798. Single measure and average measure intraclass correlations for the reviewers were 0.733 (95% confidence interval: 0.720-0.745) and 0.917 (95% confidence interval: 0.912-0.921), respectively. An exploratory factor analysis revealed six factors: history taking, physical examination, clinical reasoning, management and outcome, rhetoric, and patient relationship. In conclusion, we have developed a peer review system of medical records with high inter-rater reliability, which may enable us, with further validity analysis, to measure quality of patient care as an outcome evaluation of medical education in the future.

show abstract

Components of postgraduate competence: analyses of thirty years of longitudinal data

Abstract: Psychometric support for measuring the components of Knowledge and Clinical Capabilities, and Professionalism provides an instrument to empirically evaluate educational outcomes to medical educators who are in search of such a tool.

Cited by 32 publications

References 22 publications

Impact of performance in a mandatory postgraduate surgical examination on selection into specialty training

Impact of performance in a mandatory postgraduate surgical examination on selection into specialty training

Internal structure of mini-CEX scores for internal medicine residents: factor analysis and generalizability

Development of a Peer Review System Using Patient Records for Outcome Evaluation of Medical Education: Reliability Analysis

Contact Info

Product

Resources

About