A Valid and Reliable Assessment Tool for Remote Simulation-Based Ultrasound-Guided Regional Anesthesia

Laurent, David Burckett-St; Niazi, Ahtsham U.; Cunningham, Melissa; Jaeger, Melanie; Abbas, Sherif; McVicar, Jason; Chan, Vincent

doi:10.1097/aap.0000000000000165

Cited by 26 publications

(14 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Chin et al validated a checklist and GRS, modified from a surgical skills assessment tool for ultrasound‐guided supraclavicular block. Laurent et al proposed construct validity for GRS in the context of simulation‐based ultrasound‐guided regional anaesthesia. However, use of the checklist failed to demonstrate construct validity in a simulation setting, and the reported IRR was less than the acceptable limit of 0.8.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Construct validity of a novel assessment tool for ultrasound‐guided axillary brachial plexus block

et al. 2016

View full text Add to dashboard Cite

The purpose of this study was to examine the construct validity and reliability of a novel metrics-based assessment tool, previously developed for ultrasound-guided axillary brachial plexus block. Five expert and eight novice anaesthetists performed a total of 18 ultrasound-guided axillary brachial plexus blocks on the same number of patients. A trained investigator video-taped procedures according to a pre-defined protocol. Two trained consultant anaesthetists independently scored the videos using the assessment tool. Compared with novices, experts completed more steps (mean 41.0 vs. 33.1, p = 0.001), had fewer procedural errors (2.8 vs. 7.9, p < 0.0001), had fewer critical errors (0.8 vs. 1.3, p = 0.030), and fewer total errors (3.5 vs. 9.1, p < 0.0001). The mean inter-rater reliability for scoring of experts' performance was 0.91, for novices' performance was 0.84, and for all performance combined (n = 18) was 0.88. This assessment tool is valid, and discriminates reliably between expert and novice performance for placement of ultrasound-guided axillary brachial plexus blocks.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Our study differs from these four studies in a number of important ways. Naik et al, Chin et al, Laurent et al and Chuan et al employed Likert scales. The use of Likert scales introduces subjectivity and poor reliability in the assessment process .…”

Section: Discussionmentioning

confidence: 99%

Construct validity of a novel assessment tool for ultrasound‐guided axillary brachial plexus block

et al. 2016

View full text Add to dashboard Cite

show abstract

“…Reported as number of returns (proportion of cohort). The cohorts correspond to low (normalised checklist score < 50%, raw GRS score < 15), intermediate (normalised checklist score 50-75%, raw GRS score [15][16][17][18][19][20][21][22][23][24][25] and high (normalised checklist score > 75%, raw GRS score > 25). Number (proportion).…”

Section: Discussionmentioning

confidence: 99%

Design and validation of the Regional Anaesthesia Procedural Skills Assessment Tool

et al. 2015

View full text Add to dashboard Cite

SummaryThe aim of this study was to create and evaluate the validity, reliability and feasibility of the Regional Anaesthesia Procedural Skills tool, designed for the assessment of all peripheral and neuraxial blocks using all nerve localisation techniques. The first phase was construction of a 25-item checklist by five regional anaesthesia experts using a Delphi process. This checklist was combined with a global rating scale to create the tool. In the second phase, initial validation by 10 independent anaesthetists using a test-retest methodology was successful (Cohen kappa ≥ 0.70 for interrater agreement, scores between test to retest, paired t-test, p > 0.12). In the third phase, 70 clinical videos of trainees were scored by three blinded international assessors. The RAPS tool exhibited face validity (p < 0.026), construct validity (p < 0.001), feasibility (mean time to score < 3.9 min), and overall reliability (intraclass correlation coefficient 0.80 (95% CI 0.67-0.88)). The Regional Anaesthesia Procedural Skills tool used in this study is a valid and reliable assessment tool to score the performance of trainees for regional anaesthesia.

show abstract

“…Whilst both studies found evidence for construct validity, Wong and colleagues 53 found that the inter-rater reliability was overall poor (ICC 0.44, single-measure ICC model 2A, 1) despite piloting, modifications to the descriptors, and assessor training. In contrast, Burckett-St. Laurent and colleagues 54 found moderate reliability for both checklist and GRS in the clinical setting, but less in the simulation environment (ICC 0.6e0.8, average-measure ICC model not specified). The difference in reliabilities reported by these two studies is large and likely because of the use of different ICC models.…”

Section: Combined Checklists and Grssmentioning

confidence: 97%

Competency-based assessment tools for regional anaesthesia: a narrative review

Chuan

Wan

Royse

et al. 2018

British Journal of Anaesthesia

View full text Add to dashboard Cite

Competency-based assessment tools are used in regional anaesthesia to measure the performance of study participants, trainees, and consultants. This narrative review was performed to appraise currently published assessment tools for regional anaesthesia. A literature search found 397 citations of which 28 peer-reviewed studies met the inclusion criteria of primary psychometric evaluation of assessment tools for regional anaesthesia. The included studies were diverse in the type of assessment and the skill set being assessed. The types of assessments included multiple-choice questions, hand-motion analysis, cumulative sum, visuospatial and psychomotor screening, checklists, and global rating scales. The skill sets that were assessed included holistic regional anaesthesia technical and non-technical performance observed at the bedside, to isolated part-tasks, such as needle tip visualisation under ultrasound. To evaluate validity and reliability, we compared the studies against published medical education consensus statements on ideal assessment tools. We discuss the relative merits of different tools when used to assess regional anaesthesia, the importance of psychometrically robust assessment tools in competency-based anaesthesia education, and directions for future education research in regional anaesthesia.

show abstract

A Valid and Reliable Assessment Tool for Remote Simulation-Based Ultrasound-Guided Regional Anesthesia

Abstract: This is the first study to demonstrate the validity and reliability of a Global Rating Scale assessment tool for use in UGRA simulation training. Although the checklist may require further refinement, the Global Rating Scale can be used for remote and on-site assessment of UGRA skills.

Cited by 26 publications

References 20 publications

Construct validity of a novel assessment tool for ultrasound‐guided axillary brachial plexus block

Construct validity of a novel assessment tool for ultrasound‐guided axillary brachial plexus block

Design and validation of the Regional Anaesthesia Procedural Skills Assessment Tool

Competency-based assessment tools for regional anaesthesia: a narrative review

Contact Info

Product

Resources

About