Musculoskeletal Ultrasound Objective Structured Clinical Examination: An Assessment of the Test

Abstract

Objective. To determine the reliability and validity of an objective structured clinical examination (OSCE) for musculoskeletal ultrasound (MSUS).Methods. A 9-station OSCE was administered to 35 rheumatology fellows trained in MSUS and to 3 expert faculty (controls). Participants were unaware of joint health (5 diseased/4 healthy). Faculty assessors (n ‫؍‬ 9) graded image quality with predefined checklists and a 0 -5 global rating, blinded to who performed the study. Interrater reliability, correlation between a written multiple choice question examination (MCQ) and OSCE performance, and comparison of fellow OSCE results with those of the faculty were measured to determine OSCE reliability, concurrent validity, and construct validity. Results. Assessors' interrater reliability was good (intraclass correlation coefficient [ICC] 0.7). Score reliability was good in the normal wrist and ankle stations (ICC 0.7) and moderate in the abnormal wrist and ankle stations (ICC 0.4). MCQ grades significantly correlated with OSCE grades (r ‫؍‬ 0.52, P < 0.01). The fellows in the bottom quartile of the MCQ scored 3.07 on the OSCE, significantly worse than the top quartile fellows (3.32) and the faculty (3.29; P < 0.01). Scores also significantly discriminated bottom quartile fellows from faculty in the normal wrist and ankle stations (3.38 versus 3.78; P < 0.01), but not in the abnormal stations (3.37 versus 3.49; P ‫؍‬ 0.08). Conclusion. MSUS OSCE is a reliable and valid method for evaluation of MSUS skill. Normal joint assessment stations are more reliable than abnormal joint assessment stations and better discriminate poorly performing fellows from faculty. Therefore, MSUS OSCE with normal joints can be used for the assessment of MSUS skill competency.

Desempeño del examen clínico objetivo estructurado como instrumento de evaluación en la certificación nacional como reumatólogo

“…Por una parte, se correlacionó el desempeño del ECOE con el de la parte teórica del examen de certificación. En la literatura se han descrito correlaciones significativas entre el ECOE y las evaluaciones teóricas con valores que oscilan entre 0,59 y 0,71 [16][17][18] , si bien existe escasa información en el ámbito reumatológico 11,12 ; un reporte reciente publicado por Kissin et al describen una correlación de 0,52 (p ≤ 0,001) entre un circuito de n9 estaciones diseñado para evaluar habilidades de ultrasonido musculo-esquelético y su correspondiente parte teórica 12 ; por el contrario, Raj et al no encontraron una correlación significativa entre los conocimientos teórico y práctico, cuando este último se evaluó mediante 2 estaciones de exploración de mano y rodilla y se aplicó a estudiantes de medicina 19 . Encontramos una correlación de 0,44 en el año 2013, mientras que no hubo correlación en el año 2014.…”

“…En México, se emplea desde 1996 como examen sumativo en la Facultad de Medicina de la Universidad Autónoma de México y como la fase práctica del examen profesional de la Licenciatura desde el año 2002 7 . Además, el ECOE ha mostrado también buenos resultados para evaluar a médicos generales 8 , cirujanos 9 , pediatras 10 , reumatólogos 11 y habilidades de ultrasonido músculo-esquelético 12 .…”

Musculoskeletal Ultrasound and Anatomy: Comment on the Article by Navarro‐Zarza et al

“…As a group, we (the Ultrasound School of North American Rheumatologists [USSONAR]) have participated in the training of over 130 rheumatology fellows in the use of MSUS. Fellows participating in the USSONAR program demonstrated proficiency in musculoskeletal anatomy on both practical and written examinations (2,3). We are therefore confident that every fellow who completes MSUS training would be able to identify Lister tubercle of the radius, the structure that only 29% of the participants correctly identified in the study by Navarro-Zarza et al…”

“…Indeed, one author (CH-D) heads the MSUS laboratory at a major academic institution, another author (PV-O) had a full year of training in MSUS, and yet another author (JJC) used dynamic MSUS in the 1980s (1); sensed the enormous utility that MSUS could have in fellow training, patient care, and research (2); and saluted the first MSUS course held in the US taught by and intended for rheumatologists (3). However, as we reviewed the dismal results of the practical anatomy examinations we took in 7 American countries, the inescapable conclusion was reached that, with the current level of anatomic knowledge, rheumatologists in general are not yet optimally prepared to become competent practitioners of MSUS.…”

“…In contrast, there is no indication that the prevalence of rheumatoid factor (RF) and anti-citrullinated protein antibodies (ACPA) may be higher in smokers than in the general population. Recently, 2 articles have suggested smoking as a potential risk factor for the radiologic severity of RA (3,4). However, after adjustment, it has been shown that the effect of smoking on joint damage was mediated via ACPA and that smoking was not an independent risk factor for radiologic progression in RA.…”