Assessment of competence in local anaesthetic thoracoscopy: development and validity investigation of a new assessment tool

Abstract

Background: The aims of the study were to develop an assessment tool in local anaesthetic thoracoscopy (LAT), investigate validity evidence, and establish a pass/fail standard.Methods: Validity evidence for the assessment tool was gathered using the unified Messick framework.The tool was developed by five experts in respiratory medicine and medical education. Doctors with varying experience performed two consecutive procedures in a standardized, simulation-based setting using a newly developed thorax/lung silicone model. Performances were video-recorded and assessed by four expert raters using the new tool. Contrasting groups' method was used to set a pass/fail standard.Results: Nine novices and 8 experienced participants were included, generating 34 recorded performances and 136 expert assessments. The tool had a high internal consistency (Cronbach's alpha =0.94) and high inter-rater reliability (Cronbach's alpha =0.91). The total item score significantly correlated with the global score (r s =0.86, P<0.001). Participants' first performance correlated to second performance (test-retest reliability) with a Pearson's r of 0.93, P<0.001. Generalisability (G) study showed a G-coefficient of 0.92 and decision (D) study estimated that one performance assessed by two raters or four performances assessed by one rater are needed to reach an acceptable reliability, i.e., G-coefficient >0.80. The tool was able to discriminate between the two groups in both performances: experienced mean score =30.8±4.2; novice mean score =15.8±2.3, P<0.001. Pass/fail standard was set at 22 points. Conclusions:The newly developed assessment tool showed solid evidence of validity and can be used to ensure competence in LAT.

Interventional Pulmonary Fellowship Training: End of the Beginning

“…This is to ensure sufficient institutional expertise and resources to provide the required procedural exposure to the IP fellow; however, it does not stipulate the number of procedures the fellow must complete or methods to objectively assess procedural competence ( 4 ). This seems contrary to the current philosophy of competency-based medical education in a time when validated assessment tools and learning curves already exist for some IP procedures (i.e., rigid bronchoscopic intubation, navigation bronchoscopy, endobronchial ultrasound transbronchial needle aspiration) ( 8 – 10 ). A significant challenge will be how to transition requirements from a procedural volume model to perhaps some combination or standalone summative assessment tool(s).…”

Education in Interventional Pulmonology: How We Got Here and Where to Go from Here

“…Medical thoracoscopy is used to visualize the pleural space and obtain tissue samples from the parietal pleural in patients with exudative effusions of unknown etiology. The local anesthetic thoracoscopy (LAT) tool has been recently developed and validated to assess competency in performing medical thoracoscopy [49]. The tool is an eight-item assessment that tests technical skill throughout the procedure, which for the purposes of the assessment is performed on a standardized model.…”