A Novel Tool for Teaching Cardiac Point-of-Care Ultrasound: An Exploratory Application of the Design-Based Research Approach*

Abstract

Objectives: To develop a competency-based educational tool for cardiac point-of-care ultrasound image interpretation using the design-based research approach. Design: In accordance with design-based research, multiple methods were used to determine that learners would benefit from achieving competency in cardiac point-of-care ultrasound image interpretation before they attempted it at the bedside. A competency-based educational tool was designed and underwent a rapid prototyping process with experts. Evaluative data was used to redesign and reevaluate the educational tool with a new group of learners to improve its effectiveness. Setting: Large, university-based children’s hospital. Subjects: Pediatric critical care attending physicians, fellow physicians and advanced practice providers, and pediatric resident physicians. Interventions: Integrating mastery learning and deliberate practice as theoretical frameworks, the authors designed an online “Image Library” composed of 90 questions of cardiac point-of-care ultrasound image clips with varying degrees of pathology that were organized into three levels of difficulty. Learners answered a set of 10 questions, learned from feedback, and repeated an additional set in the same difficulty until achieving a predefined level of mastery. Measurements and Main Results: Two learning cycles were implemented with a total of 41 learners. Forty learners (98%) were able to demonstrate mastery on the Image Library and required a range of 30 to 210 questions to do so. On a 10-question final assessment, learners scored an average of 72%, while experts and novices scored 83% and 17%, respectively. On a 5-point scale, learners rated the quality of the Image Library for facilitation of learning (median 5), enjoyment of learning (5), and useful knowledge and skills to improve clinical practice (5). Conclusions: An effective and innovative tool for teaching cardiac point-of-care ultrasound image interpretation was developed using the design-based research approach. Our results demonstrate the importance of individualized learning timelines for ultrasound image interpretation.

Learning Pediatric Point-of-Care Ultrasound

“…This is in contrast to the other applications, where even though there may be different methods to acquire a view, ultimately, the video and still images present the anatomy in a similar way. We also found that poor cardiac function and cases with abnormal ventricles were among the most difficult to diagnose, which is consistent with prior research in this area 11 . This may be related to the low a priori skill in this type of pathology from bedside encounters and relatively few of these case examples in education interventions.…”

“…We also found that poor cardiac function and cases with abnormal ventricles were among the most difficult to diagnose, which is consistent with prior research in this area. 11 This may be related to the low a priori skill in this type of pathology from bedside encounters and relatively few of these case examples in education interventions. Other diagnostically challenging cases were subtle imaging findings such as foreign bodies with respective acoustic shadowing/reverberation or the presence of lung sliding/lung point suggesting a pneumothorax.…”

“…20 Using this approach, participants could learn similarities and differences between diagnoses, identify weaknesses, and build up a global representation of imaging diagnoses. [21][22][23] In line with principles of mastery learning, 8,10,11,24 a revised version of the original previously published platform permitted participants to do as many cases as needed to reach a specific performance benchmark derived from a consensus opinion of an international pediatric POCUS community (https://p2network.com). That is, 75 (81.5%) of 92 member respondents reported an achievement of a combined 90% accuracy (overall performance), 90% sensitivity (performance on normal cases), and 90% specificity (performance on normal cases) as a reasonable and safe performance benchmark for physician credentialing.…”

“…However, the baseline pathology rate in pediatrics is relatively low in many POCUS applications; thus, the validity of current case-number requirements that represent proficiency for the image interpretation subcompetency may be limited. As a result, some studies have tried to characterize image interpretation proficiency in pediatric POCUS applications using collections of bedside ultrasound derived images and assessing participant accuracy and/or number of cases completed to a performance standard 9,11,17 . However, these studies are limited by the relatively small number of participants at a single site, a small number of cases that can result in participants memorizing case solutions, expert-derived priorities of case selection, qualitative estimates of case difficulty, a subjective selection of a performance standard, and/or the use of modeled rather than raw data to extrapolate number of cases needed to achieve a performance standard 9–11,17 .…”

“…As a result, some studies have tried to characterize image interpretation proficiency in pediatric POCUS applications using collections of bedside ultrasound derived images and assessing participant accuracy and/or number of cases completed to a performance standard. 9,11,17 However, these studies are limited by the relatively small number of participants at a single site, a small number of cases that can result in participants memorizing case solutions, expert-derived priorities of case selection, qualitative estimates of case difficulty, a subjective selection of a performance standard, and/or the use of modeled rather than raw data to extrapolate number of cases needed to achieve a performance standard. [9][10][11]17 Thus, given the increasing priority of POCUS in pediatric emergency practice, research is urgently needed to determine how well the current recommended standards represent physician proficiency across different learning groups and varied educational environments, 13 and how specific case-difficulty measures should be incorporated in the learning and assessment of this skill.…”