Assessment of procedural skills in anaesthesia

Abstract

A key aspect of the practice of anaesthesia is the ability to perform practical procedures efficiently and safely. Decreased working hours during training, an increasing focus on patient safety, and greater accountability have resulted in a paradigm shift in medical education. The resulting international trend towards competency-based training demands robust methods of evaluation of all domains of learning. The assessment of procedural skills in anaesthesia is poor compared with other domains of learning and has fallen behind surgical fields. Logbooks and procedure lists are best suited to providing information regarding likely opportunities within training programmes. Retrospective global scoring and direct observation without specific criteria are unreliable. The current best evidence for a gold standard for assessment of procedural skills in anaesthesia consists of a combination of previously validated checklists and global rating scales, used prospectively by a trained observer, for a procedure performed in an actual patient. Future research should include core assessment parameters to ensure methodological rigor and facilitate robust comparisons with other studies: (i) reliability, (ii) validity, (iii) feasibility, (iv) cost-effectiveness, and (v) comprehensiveness with varying levels of difficulty. Simulation may become a key part of the future of formative and summative skills assessment in anaesthesia; however, research is required to develop and test simulators that are realistic enough to be suitable for use in high-stakes evaluation.

Assessment of procedural skills training and performance in anesthesia using cumulative sum analysis (cusum)

“…5 Owing to the dependency of patients' outcomes on a surgeon's technical skills, research into this area has been pioneered in surgery. 6 Several assessment tools have now been developed and validated for use on surgical trainees outside the operating room.…”

“…5,[27][28][29][30] In one study, cusum was used to determine the amount of training required to achieve competency in spinal ultrasound. 27 The conclusion was that 20 attempts and a coaching session were not sufficient to teach the relevant skills, and that this should inform the planning of future educational sessions and workshops.…”

Review article: Assessment in anesthesiology education

“…13,14 Combining various simulation modalities (e.g., standardized patients, task trainers, electromechanical mannequins) allows for a broader modelling of practice situations, making it possible to measure the multiple abilities or competencies required in anesthesia practice. [15][16][17][18] Nevertheless, while some very innovative scenario designs have been forwarded, there remains the need to ensure that student, resident, or practitioner assessments generate resulting scores or decisions that are meaningful and accurate. The qualities of ''good'' assessments are discussed in the next section.…”

“…35,36 As mentioned previously, the long history of simulation in anesthesiology 37 coupled with advances in technology has effectively broadened the potential assessment domain for the specialty. 18 This affiliation has allowed for the measurement of both procedural 17 and non-technical skills such as communication, situational awareness, teamwork, and professionalism. 38 A thorough review of the use of simulation for assessment in anesthesiology can be found elsewhere.…”

“…More importantly, the subjectivity of the rating process can often be controlled through the specification of behavioural benchmarks and the incorporation of structured rater training regimes. 17 In many situations, the subjectivity of the ratings is simply a function of the raters not knowing exactly what is being measured.…”