Assessment of competences in rheumatology training: results of a systematic literature review to inform EULAR points to consider

Abstract

ObjectiveTo summarise the literature on the assessment of competences in postgraduate medical training.MethodsA systematic literature review was performed within a EULAR taskforce on the assessment of competences in rheumatology training and other related specialities (July 2019). Two searches were performed: one search for rheumatology and one for related medical specialities. Two reviewers independently identified eligible studies and extracted data on assessment methods. Risk of bias was assessed using the medical education research study quality instrument.ResultsOf 7335 articles in rheumatology and 2324 reviews in other specialities, 5 and 31 original studies were included, respectively. Studies in rheumatology were at variable risk of bias and explored only direct observation of practical skills (DOPS) and objective structured clinical examinations (OSCEs). OSCEs, including clinical, laboratory and imaging stations, performed best, with a good to very good internal consistency (Cronbach’s α=0.83–0.92), and intrarater reliability (r=0.80–0.95). OSCEs moderately correlated with other assessment tools: r=0.48 vs rating by programme directors; r=0.2–0.44 vs multiple-choice questionnaires; r=0.48 vs DOPS. In other specialities, OSCEs on clinical skills had a good to very good inter-rater reliability and OSCEs on communication skills demonstrated a good to very good internal consistency. Multisource feedback and the mini-clinical evaluation exercise showed good feasibility and internal consistency (reliability), but other data on validity and reliability were conflicting.ConclusionDespite consistent data on competence assessment in other specialities, evidence in rheumatology is scarce and conflicting. Overall, OSCEs seem an appropriate tool to assess the competence of clinical skills and correlate well with other assessment strategies. DOPS, multisource feedback and the mini-clinical evaluation exercise are feasible alternatives.

Intra-articular therapies: patient preferences and professional practices in European countries

“…Both professional background and specific training differ across Europe. This could be due to the marked variability in aims, structure, and content of postgraduate training programmes [14][15][16][17][18][19][20]. In a European survey, residents and young rheumatologists felt quite confident in performing knee aspiration (average 9/10), although confidence was statistically greater in those who had received formal training [19].…”

EULAR portfolio for Rheumatology training: a EULAR School of Rheumatology initiative

“… 6 Cut-off points of the MERSQI (score range 5–18) are not clearly defined, but based on previous studies, the following cut-off points were used:<10 high risk of bias; 10–11 moderate risk of bias, ≥12 low risk of bias. 7 Studies were too heterogeneous to allow pooling; descriptive results are therefore presented.…”

Strategies for the assessment of competences during rheumatology training across Europe: results of a qualitative study

“…These statements were developed by the medical educator (CH) based on general principles and medical education literature and a systematic literature review on the assessment of competences. 12 Quotations were thoroughly collected from this part for each group. Participants were finally asked to specify any aspect which had, in their view, been omitted in order to ensure comprehensiveness of the final picture and to assess whether data saturation had been reached.…”