Workplace-Based Assessments Using Pediatric Critical Care Entrustable Professional Activities

Emke, Amanda R.; Park, Yoon Soo; Srinivasan, Shardha; Tekian, Ara

doi:10.4300/jgme-d-18-01006.1

Cited by 14 publications

(20 citation statements)

References 32 publications

(60 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…[7][8][9][10] Our baseline rate is similar to that of other preintervention cohorts and exceeds the relative gain made in other studies implementing improvements to increase the frequency of feedback. [9][10][11] The locally tailored solution we developed was more effective for our department but is perhaps less generalizable than the interventions reported in the larger studies. 9,10 Potential limitations of the study include the instability of the data at baseline, which decreases some statistical confidence in the results.…”

Section: Discussionmentioning

confidence: 91%

Feedback Frequency in Competence by Design: A Quality Improvement Initiative

Arnstead

Campisi

Takahashi

et al. 2020

Journal of Graduate Medical Education

View full text Add to dashboard Cite

Background Otolaryngology–head and neck surgery is in the first wave of residency training programs in Canada to adopt Competence by Design (CBD), a model of competency-based medical education. CBD is built on frequent, low-stakes assessments and requires an increase in the number of feedback interactions. The University of Toronto otolaryngology–head and neck surgery residents piloted the CBD model but were completing only 1 assessment every 4 weeks, which was insufficient to support CBD. Objective This project aimed to increase assessment completion to once per resident per week using quality improvement methodology. Methods Stakeholder engagement activities had residents and faculty characterize barriers to assessment completion. Brief electronic assessment forms were completed by faculty on residents' personal mobile devices in face-to-face encounters, and the number completed per resident was tracked for 10 months during the 2016–2017 pilot year. Response to the intervention was analyzed using statistical process control charts. Results The first bundled intervention—a rule set dictating which clinical instance should be assessed, combined with a weekly reminder implemented for 10 weeks—was unsuccessful in increasing the frequency of assessments. The second intervention was a leaderboard, designed on an audit-and-feedback system, which sent weekly comparison e-mails of each resident's completion rate to all residents and the program director. The leaderboard demonstrated significant improvement from baseline over 10 weeks, increasing the assessment completion rate from 0.22 to 2.87 assessments per resident per week. Conclusions A resident-designed audit-and-feedback leaderboard system improved the frequency of CBD assessment completion.

show abstract

Section: Discussionmentioning

confidence: 91%

Feedback Frequency in Competence by Design: A Quality Improvement Initiative

Arnstead

Campisi

Takahashi

et al. 2020

Journal of Graduate Medical Education

View full text Add to dashboard Cite

show abstract

“…Next steps could potentially focus on a narrower scope on skills by observing practice activities and using a checklist of foundational steps in each EPA. 26,27 This work comes with limitations. Although there is interest in EPAs in pharmacy education, the number of experts who have experience using EPAs in the experiential setting are limited.…”

Section: Discussionmentioning

confidence: 99%

Development of an Entrustment-Supervision Assessment Tool for Pharmacy Experiential Education Using Stakeholder Focus Groups

Jarrett

Goliak

Haines

et al. 2022

American Journal of Pharmaceutical Education

View full text Add to dashboard Cite

Objective. To devise a pharmacy-specific, expanded Entrustable Professional Activities (EPA) entrustment-supervision scale, which frames preceptor ratings of entrustment in a prospective and retrospective manner for use in experiential learning settings. Methods. A series of focus group sessions were conducted to solicit expert opinion to develop and refine two entrustment-supervision scales. Purposive sampling was used to identify experts from different professional groups (physicians, pharmacy experiential administrators, and pharmacy practice faculty) and who had extensive knowledge regarding EPAs and at least one publication related to EPAs. Panelists were invited to participate via email. Three focus sessions were conducted by videoconferencing between June and September 2019. The primary outcome was development of a pharmacy-specific EPA entrustment-supervision assessment tool. Secondary outcomes were individual entrustment-supervision statements across five levels of the entrustment-supervision scale. Results. The focus group consisted of four pharmacy practice faculty, two experiential administrators, and one academic physician. Four concepts emerged from the focus group discussion: 1. need for more granularity in entrustmentsupervision scales; 2. limitations due to differences in licensure requirements across the United States; 3. colleges/schools of pharmacy already use expanded scales; and 4. uncertainty how to rate entrustment-supervision when a student exhibits unprofessional behavior. Conclusion.A pharmacy-specific, expanded EPA entrustment-supervision scale will be useful to support longitudinal assessment of learners in experiential settings utilizing EPAs. Determining when to use a prospective versus retrospective perspective requires further evaluation.

show abstract

“…6,7,18 However, a reported disadvantage of high-frequency assessment systems was the difficulty in collecting, handling, and interpreting large amounts of data. 6e8, 12 Further, compared to end-of-rotation assessments, high-frequency end-of-shift assessments had lower correlation with overall performance as judged by clinical competence committees. 10…”

Section: Frequencymentioning

confidence: 92%

“…Tool granularity ranges from broad aspects of performance and whole procedures down to discrete technical skills. Examples include entrustable professional activities (EPAs), 17,23,25,26 which assess an entire domain of practice, whilst sub-EPAs or observable practice activities (OPAs) 12,24 focus down on single tasks. 12 A tool often contains nested granularities.…”

Section: Granularitymentioning

confidence: 99%

“…Examples include entrustable professional activities (EPAs), 17,23,25,26 which assess an entire domain of practice, whilst sub-EPAs or observable practice activities (OPAs) 12,24 focus down on single tasks. 12 A tool often contains nested granularities. For example, the items might refer to technical skills that are clustered around competencies or sub-competencies, 9 in which case the tool assesses the coarser granularity of competencies by measuring the finer granularity of technical skills.…”

Section: Granularitymentioning

confidence: 99%

See 1 more Smart Citation

Key dimensions of innovations in workplace-based assessment for postgraduate medical education: a scoping review

Weller

Coomber

Chen

et al. 2021

British Journal of Anaesthesia

View full text Add to dashboard Cite

Background: Specialist training bodies continue to devise innovative methods of gathering information on trainee workplace performance to meet the requirements of competency-based medical education. We reviewed recent innovations in workplace-based assessment (WBA) tools to identify strengths, weaknesses, and trade-offs inherent in their design and use. Methods: In this scoping review, using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, we systematically searched databases between 2009 and 2019 for WBA tools with novel characteristics not typically seen in traditional WBAs. These included innovations in rating scales, ways of collecting information, technological innovations, ways of triggering WBAs, and approaches to compiling and using information. Results: We identified 30 innovative WBA tools whose characteristics could be categorised into seven dimensions: frequency of assessment, granularity (unit of performance assessed), coverage of the curriculum, rating method, initiation of the WBA, information use, and incentives. These dimensions had multiple interdependencies and trade-offs, often balancing generating assessment data with available resources. Philosophical stance on assessment also influenced WBA choice, for example prioritising trainee-centred learning (i.e. initiation of WBA and transparency of assessment data), perceptions of assessment and feedback as burdensome or beneficial, and holistic vs reductionist views on assessment of performance. Conclusions: Our synthesis of the literature on innovative WBAs provides a framework for categorising tool characteristics across seven dimensions, systematically teasing apart the considerations in design and use of workplace assessments. It also draws attention to the trade-offs inherent in tool design and selection, and enables a more deliberate consideration of the tool characteristics most appropriate to the local context.

show abstract

Workplace-Based Assessments Using Pediatric Critical Care Entrustable Professional Activities

Cited by 14 publications

References 32 publications

Feedback Frequency in Competence by Design: A Quality Improvement Initiative

Feedback Frequency in Competence by Design: A Quality Improvement Initiative

Development of an Entrustment-Supervision Assessment Tool for Pharmacy Experiential Education Using Stakeholder Focus Groups

Key dimensions of innovations in workplace-based assessment for postgraduate medical education: a scoping review

Contact Info

Product

Resources

About