Assessment of Junior Doctor performance: a validation study

Carr, Sandra; Celenza, Antonio; Lake, Fiona

doi:10.1186/1472-6920-13-129

Cited by 10 publications

(8 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We measured doctors’ performance using a four-item clinical performance measurement scale (α = 0.76) adopted from Carr et al (2013) . The scale included items such as ‘clinical assessment and patient management,’ ‘procedural skills,’ and ‘emergency management.’ The doctor’s performance was rated from 1 (performance below the minimum acceptable level) to 5 (performance consistently far exceeded expectations).…”

Section: Methodsmentioning

confidence: 99%

Moderated Mediation Model of Interrelations between Workplace Romance, Wellbeing, and Employee Performance

Khan

Usman

et al. 2017

Front. Psychol.

View full text Add to dashboard Cite

In this study, first we examined the effect of workplace romance on employee job performance, and the mediatory role of psychological wellbeing in the relationship between workplace romance and employee performance. Then we tested the moderating effects of gender and workplace romance type – lateral or hierarchical – on the indirect effect of workplace romance on employee performance. Based on a survey of 311 doctors from five government teaching hospitals in Pakistan, we used structural equation modeling and bootstrapping to test these relationships. This study reveals that psychological wellbeing significantly fully mediates the positive relationship between workplace romance and job performance. Moreover, multi-group analysis shows that gender moderates the indirect effect of workplace romance on employee performance, where the indirect effect of workplace romance on employee performance is stronger for male participants. This study carries important implications, particularly for the policy makers and managers of healthcare sector organizations.

show abstract

Section: Methodsmentioning

confidence: 99%

Moderated Mediation Model of Interrelations between Workplace Romance, Wellbeing, and Employee Performance

Khan

Usman

et al. 2017

Front. Psychol.

View full text Add to dashboard Cite

show abstract

“…In a validation study of the JDAT, we identified a Cronbach Alpha of 0.883 for the 10 item scale and identified that two principal components of junior doctor performance are being assessed rather than the commonly reported three [ 15 ]. Cronbach Alphas were 0.829 for the 6 item ‘Clinical Management subscale’ and 0.834 for the 4 item ‘Communication subscale’, indicating good internal consistency and reliability of the instrument in its entirety and for both subscales.…”

Section: Methodsmentioning

confidence: 99%

Relationships between academic performance of medical students and their workplace performance as junior doctors

et al. 2014

Self Cite

View full text Add to dashboard Cite

BackgroundLittle recent published evidence explores the relationship between academic performance in medical school and performance as a junior doctor. Although many forms of assessment are used to demonstrate a medical student’s knowledge or competence, these measures may not reliably predict performance in clinical practice following graduation.MethodsThis descriptive cohort study explores the relationship between academic performance of medical students and workplace performance as junior doctors, including the influence of age, gender, ethnicity, clinical attachment, assessment type and summary score measures (grade point average) on performance in the workplace as measured by the Junior Doctor Assessment Tool.ResultsThere were two hundred participants. There were significant correlations between performance as a Junior Doctor (combined overall score) and the grade point average (r = 0.229, P = 0.002), the score from the Year 6 Emergency Medicine attachment (r = 0.361, P < 0.001) and the Written Examination in Year 6 (r = 0.178, P = 0.014). There was no significant effect of any individual method of assessment in medical school, gender or ethnicity on the overall combined score of performance of the junior doctor.ConclusionPerformance on integrated assessments from medical school is correlated to performance as a practicing physician as measured by the Junior Doctor Assessment Tool. These findings support the value of combining undergraduate assessment scores to assess competence and predict future performance.

show abstract

“…Comparable evaluations from the past of supervisors who rated trainees’ competencies have made similar observations to those of this current study, as identified in our previous review [ 17 ]. Indeed, another more recent study of a similar Australian junior doctor population also found variation in the domain constructs of what was assessed compared to the domains expected to be assessed [ 29 ]. Moreover, from an Australian perspective, other evaluative research has identified concerns about the assessment of a similar junior doctor population in Australia [ 30 – 32 ], with observations indicating “that the tools and processes being used to monitor and assess junior doctor performance could be better” [ 32 ].…”

Section: Discussionmentioning

confidence: 99%

Construct validation of judgement-based assessments of medical trainees’ competency in the workplace using a “Kanesian” approach to validation

2015

View full text Add to dashboard Cite

BackgroundEvaluations of clinical assessments that use judgement-based methods have frequently shown them to have sub-optimal reliability and internal validity evidence for their interpretation and intended use. The aim of this study was to enhance that validity evidence by an evaluation of the internal validity and reliability of competency constructs from supervisors’ end-of-term summative assessments for prevocational medical trainees.MethodsThe populations were medical trainees preparing for full registration as a medical practitioner (74) and supervisors who undertook ≥2 end-of-term summative assessments (n = 349) from a single institution. Confirmatory Factor Analysis was used to evaluate assessment internal construct validity. The hypothesised competency construct model to be tested, identified by exploratory factor analysis, had a theoretical basis established in workplace-psychology literature. Comparisons were made with competing models of potential competency constructs including the competency construct model of the original assessment. The optimal model for the competency constructs was identified using model fit and measurement invariance analysis. Construct homogeneity was assessed by Cronbach’s α. Reliability measures were variance components of individual competency items and the identified competency constructs, and the number of assessments needed to achieve adequate reliability of R > 0.80.ResultsThe hypothesised competency constructs of “general professional job performance”, “clinical skills” and “professional abilities” provides a good model-fit to the data, and a better fit than all alternative models. Model fit indices were χ2/df = 2.8; RMSEA = 0.073 (CI 0.057-0.088); CFI = 0.93; TLI = 0.95; SRMR = 0.039; WRMR = 0.93; AIC = 3879; and BIC = 4018). The optimal model had adequate measurement invariance with nested analysis of important population subgroups supporting the presence of full metric invariance. Reliability estimates for the competency construct “general professional job performance” indicated a resource efficient and reliable assessment for such a construct (6 assessments for an R > 0.80). Item homogeneity was good (Cronbach’s alpha = 0.899). Other competency constructs are resource intensive requiring ≥11 assessments for a reliable assessment score.ConclusionInternal validity and reliability of clinical competence assessments using judgement-based methods are acceptable when actual competency constructs used by assessors are adequately identified. Validation for interpretation and use of supervisors’ assessment in local training schemes is feasible using standard methods for gathering validity evidence.

show abstract

Assessment of Junior Doctor performance: a validation study

Cited by 10 publications

References 15 publications

Moderated Mediation Model of Interrelations between Workplace Romance, Wellbeing, and Employee Performance

Moderated Mediation Model of Interrelations between Workplace Romance, Wellbeing, and Employee Performance

Relationships between academic performance of medical students and their workplace performance as junior doctors

Construct validation of judgement-based assessments of medical trainees’ competency in the workplace using a “Kanesian” approach to validation

Contact Info

Product

Resources

About