Design and validation of the Regional Anaesthesia Procedural Skills Assessment Tool

Abstract

SummaryThe aim of this study was to create and evaluate the validity, reliability and feasibility of the Regional Anaesthesia Procedural Skills tool, designed for the assessment of all peripheral and neuraxial blocks using all nerve localisation techniques. The first phase was construction of a 25-item checklist by five regional anaesthesia experts using a Delphi process. This checklist was combined with a global rating scale to create the tool. In the second phase, initial validation by 10 independent anaesthetists using a test-retest methodology was successful (Cohen kappa ≥ 0.70 for interrater agreement, scores between test to retest, paired t-test, p > 0.12). In the third phase, 70 clinical videos of trainees were scored by three blinded international assessors. The RAPS tool exhibited face validity (p < 0.026), construct validity (p < 0.001), feasibility (mean time to score < 3.9 min), and overall reliability (intraclass correlation coefficient 0.80 (95% CI 0.67-0.88)). The Regional Anaesthesia Procedural Skills tool used in this study is a valid and reliable assessment tool to score the performance of trainees for regional anaesthesia.

Development of performance and error metrics for ultrasound-guided axillary brachial plexus block

“…For ultrasound-guided nerve blocks, checklists and global rating scales (GRS) have been developed and examined for validity. Both Cheung et al17 and Chuan et al18 developed a “one-size-fits-all” assessment tool for UGRA to include checklist and GRS and validated the tool by using a modified Delphi method. GRS as used in these two studies permits a degree of subjectivity, which may influence the assessment outcome.…”

Reliability of a faculty evaluated scoring system for anesthesiology resident applicants (Original Investigation)

“…Reliability and consistency of faculty scores was assessed using the intraclass coefficient one-way random effects model with score averaging (ICC) [8]. The ICC is a measure of the reliability of measurements or ratings where two or more raters rate a number of study subjects and has been increasingly used in the anesthesia literature for assessment of reliability of educational and assessment tools [9][10][11]. The ICC determines scores of b 0.4 as poor, 0.4-0.59 as fair, 0.6-0.74 as good and N 0.75 as excellent agreement (Table 1).…”

Safe performance of peripheral regional anaesthesia: the significance of ultrasound guidance

“…The other, perhaps more pertinent barrier, is that ultrasound-guided thoracic paravertebral block is viewed as an advanced regional anaesthetic technique, being technically challenging, time consuming, and carrying with it not insignificant risks such as pneumothorax. As modern regional anaesthesia is now highly subspecialised, and indepth teaching and training are required for competence in each block [22], it is becoming more difficult to deliver training in techniques such as ultrasound-guided thoracic paravertebral block within training programs that have an unpredictable clinical workload [23]. Should we, therefore, limit the scope of blocks taught to anaesthetists who have not completed a regional anaesthesia fellowship [24]?…”