Queen’s Simulation Assessment Tool

Hall, Andrew K.; Dagnone, Jeffrey Damon; Lacroix, Lauren; Pickett, William; Klinger, Don A.

doi:10.1097/sih.0000000000000076

Cited by 35 publications

(23 citation statements)

References 35 publications

(14 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…High-fidelity simulation (HFS) offers the opportunity to train and assess medical students on EPA-10 related competencies. Literature on the use of HFS for assessing EPA-10 is limited; however, residents at some Canadian institutions have been effectively assessed with checklists and HFS 17 . 18 …”

Section: Discussionmentioning

confidence: 99%

Development of an Assessment for Entrustable Professional Activity (EPA) 10: Emergent Patient Management

Thompson

Leung

Green

et al. 2017

WestJEM

View full text Add to dashboard Cite

IntroductionMedical schools in the United States are encouraged to prepare and certify the entrustment of medical students to perform 13 core entrustable professional activities (EPAs) prior to graduation. Entrustment is defined as the informed belief that the learner is qualified to autonomously perform specific patient-care activities. Core EPA-10 is the entrustment of a graduate to care for the emergent patient. The purpose of this project was to design a realistic performance assessment method for evaluating fourth-year medical students on EPA-10.MethodsFirst, we wrote five emergent patient case-scenarios that a medical trainee would likely confront in an acute care setting. Furthermore, we developed high-fidelity simulations to realistically portray these patient case scenarios. Finally, we designed a performance assessment instrument to evaluate the medical student’s performance on executing critical actions related to EPA-10 competencies. Critical actions included the following: triage skills, mustering the medical team, identifying causes of patient decompensation, and initiating care. Up to four students were involved with each case scenario; however, only the team leader was evaluated using the assessment instruments developed for each case.ResultsA total of 114 students participated in the EPA-10 assessment during their final year of medical school. Most students demonstrated competence in recognizing unstable vital signs (97%), engaging the team (93%), and making appropriate dispositions (92%). Almost 87% of the students were rated as having reached entrustment to manage the care of an emergent patient (99 of 114). Inter-rater reliability varied by case scenario, ranging from moderate to near-perfect agreement. Three of five case-scenario assessment instruments contained items that were internally consistent at measuring student performance. Additionally, the individual item scores for these case scenarios were highly correlated with the global entrustment decision.ConclusionHigh-fidelity simulation showed good potential for effective assessment of medical student entrustment of caring for the emergent patient. Preliminary evidence from this pilot project suggests content validity of most cases and associated checklist items. The assessments also demonstrated moderately strong faculty inter-rater reliability.

show abstract

Section: Discussionmentioning

confidence: 99%

Development of an Assessment for Entrustable Professional Activity (EPA) 10: Emergent Patient Management

Thompson

Leung

Green

et al. 2017

WestJEM

View full text Add to dashboard Cite

show abstract

“… 5 , 28 , 29 , 37 This pilot study examined the performance of a previously validated simulation-based assessment tool (the QSAT) in a 3-station resuscitation OSCE at 5 post-graduate EM training programs across Canada. 32 Despite the challenges of standardizing the environmental testing conditions for all trainees, recruiting trainees for participation, and orienting new expert examiners at each of the five separate centres, the QSAT performed well, demonstrating its promise for high stakes or summative assessment contexts.…”

Section: Discussionmentioning

confidence: 99%

“…The QSAT was designed to be simple and modifiable in assessing specific and generalized resuscitation parameters. 32 It is unique in its basic framework in that it can be modified for specific clinical scenarios. The QSAT uses a standardized format with two components: 1) four domain scores (initial assessment, diagnostic approach, therapeutic approach, and communication skills, and 2) a single global assessment score (GAS).…”

Section: Methodsmentioning

confidence: 99%

“…Three previously developed and validated standardized emergency department resuscitation scenarios, each with a corresponding QSAT, were used. 32 All scenarios were based on core content and objectives for EM postgraduate programs, as outlined by the RCPSC. An expert panel of EM faculty, with training in high fidelity simulation-based instruction, developed the scenarios.…”

Section: Methodsmentioning

confidence: 99%

“…The Queen’s Simulation Assessment Tool (QSAT) was developed to allow for competency-based assessment of any EM resident trainee in a simulated resuscitation under a number of different clinical and team-based circumstances. 32 Building on previous validation work done at Queen’s University in competency-based simulation assessment, 32 , 33 a simulation-based OSCE was administered at five different academic training centres across Canada to evaluate postgraduate trainees’ resuscitation skills using the QSAT. This study had three primary aims.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Competency-based simulation assessment of resuscitation skills in emergency medicine postgraduate trainees – a Canadian multi-centred study

Dagnone

Hall²,

Sebok‐Syer³

et al. 2016

Can. Med. Ed. J

View full text Add to dashboard Cite

Background: The use of high-fidelity simulation is emerging as a desirable method for competency-based assessment in postgraduate medical education. We aimed to demonstrate the feasibility and validity of a multi-centre simulation-based Objective Structured Clinical Examination (OSCE) of resuscitation competence with Canadian Emergency Medicine (EM) trainees.Method: EM postgraduate trainees (n=98) from five Canadian academic centres participated in a high fidelity, 3-station simulation-based OSCE. Expert panels of three emergency physicians evaluated trainee performances at each centre using the Queen’s Simulation Assessment Tool (QSAT). Intraclass correlation coefficients were used to measure the inter-rater reliability, and analysis of variance was used to measure the discriminatory validity of each scenario. A fully crossed generalizability study was also conducted for each examination centre. Results: Inter-rater reliability in four of the five centres was strong with a median absolute intraclass correlation coefficient (ICC) across centres and scenarios of 0.89 [0.65-0.97]. Discriminatory validity was also strong (p < 0.001 for scenarios 1 and 3; p < 0.05 for scenario 2). Generalizability studies found significant variations at two of the study centres.Conclusions: This study demonstrates the successful pilot administration of a multi-centre, 3-station simulation-based OSCE for the assessment of resuscitation competence in post-graduate Emergency Medicine trainees.

show abstract

Comparison of Simulation‐based Resuscitation Performance Assessments With In‐training Evaluation Reports in Emergency Medicine Residents: A Canadian Multicenter Study

Hall

Dagnone

Moore

et al. 2017

AEM Education and Training

Self Cite

View full text Add to dashboard Cite

Objective: Simulation stands to serve an important role in modern competency-based programs of assessment in postgraduate medical education. Our objective was to compare the performance of individual emergency medicine (EM) residents in a simulation-based resuscitation objective structured clinical examination (OSCE) using the Queen's Simulation Assessment Tool (QSAT), with portfolio assessment of clinical encounters using a modified in-training evaluation report (ITER) to understand in greater detail the inferences that may be drawn from a simulation-based OSCE assessment.Methods: A prospective observational study was employed to explore the use of a multicenter simulation-based OSCE for evaluation of resuscitation competence. EM residents from five Canadian academic sites participated in the OSCE. Video-recorded performances were scored by blinded raters using the scenario-specific QSATs with domain-specific anchored scores (primary assessment, diagnostic actions, therapeutic actions, communication) and a global assessment score (GAS). Residents' portfolios were evaluated using a modified ITER subdivided by CanMEDS roles (medical expert, communicator, collaborator, leader, health advocate, scholar, and professional) and a GAS. Correlational and regression analyses were performed comparing components of each of the assessment methods.Results: Portfolio review and ITER scoring was performed for 79 residents participating in the simulation-based OSCE. There was a significant positive correlation between total OSCE and ITER scores (r = 0.341). The strongest correlations were found between ITER medical expert score and each of the OSCE GAS (r = 0.420), communication (r = 0.443), and therapeutic action (r = 0.484) domains. ITER medical expert was a significant predictor of OSCE total (p = 0.002). OSCE therapeutic action was a significant predictor of ITER total (p = 0.02)

show abstract

Queen’s Simulation Assessment Tool

Abstract: This study describes the development and validation of a novel modifiable anchored global assessment scoring tool for simulation-based OSCE assessment of resuscitation competence in postgraduate EM trainees.

Cited by 35 publications

References 35 publications

Development of an Assessment for Entrustable Professional Activity (EPA) 10: Emergent Patient Management

Development of an Assessment for Entrustable Professional Activity (EPA) 10: Emergent Patient Management

Competency-based simulation assessment of resuscitation skills in emergency medicine postgraduate trainees – a Canadian multi-centred study

Comparison of Simulation‐based Resuscitation Performance Assessments With In‐training Evaluation Reports in Emergency Medicine Residents: A Canadian Multicenter Study

Contact Info

Product

Resources

About