PurposeChange in the landscape of medical education coupled with a paradigm shift toward outcome-based training mandates the trainee to demonstrate specific predefined performance benchmarks in order to progress through training. A valid and reliable assessment tool is a prerequisite for this process. The objective of this study was to characterize ultrasound-guided axillary brachial plexus block to develop performance and error metrics and to verify face and content validity using a modified Delphi method.MethodsA metric group (MG) was established, which comprised three expert regional anesthesiologists, an experimental psychologist and a trained facilitator. The MG deconstructed ultrasound-guided axillary brachial plexus block to identify and define performance and error metrics. Experts reviewed five video recordings of the procedure performed by anesthesiologists with different levels of expertise to aid task deconstruction. Subsequently, the MG subjected the metrics to “stress testing”, a process to ascertain the extent to which the performance and error metrics could be scored objectively, either occurring or not occurring with a high degree of reliability. Ten experienced regional anesthesiologists used a modified Delphi method to reach consensus on the metrics.ResultsFifty-four performance metrics, organized in six procedural phases and characterizing ultrasound-guided axillary brachial plexus block and 32 error metrics (nine categorized as critical) were identified and defined. Based on the Delphi panel consensus, one performance metric was modified, six deleted and three added.ConclusionIn this study, we characterized ultrasound-guided axillary brachial plexus block to develop performance and error metrics as a prerequisite for outcome-based training and assessment. Delphi consensus verified face and content validity.
The purpose of this study was to examine the construct validity and reliability of a novel metrics-based assessment tool, previously developed for ultrasound-guided axillary brachial plexus block. Five expert and eight novice anaesthetists performed a total of 18 ultrasound-guided axillary brachial plexus blocks on the same number of patients. A trained investigator video-taped procedures according to a pre-defined protocol. Two trained consultant anaesthetists independently scored the videos using the assessment tool. Compared with novices, experts completed more steps (mean 41.0 vs. 33.1, p = 0.001), had fewer procedural errors (2.8 vs. 7.9, p < 0.0001), had fewer critical errors (0.8 vs. 1.3, p = 0.030), and fewer total errors (3.5 vs. 9.1, p < 0.0001). The mean inter-rater reliability for scoring of experts' performance was 0.91, for novices' performance was 0.84, and for all performance combined (n = 18) was 0.88. This assessment tool is valid, and discriminates reliably between expert and novice performance for placement of ultrasound-guided axillary brachial plexus blocks.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.