The Joint Council on Thoracic Surgery Education Coronary Artery Assessment Tool Has High Interrater Reliability

“Top Gun” Competition: Motivation and Practice Narrows the Technical Skill Gap Among New Cardiothoracic Surgery Residents

“…The average score for a pass was 4.0, whereas fail was 3.0. This finding implies that the raters used a pass-fail cutoff at approximately 3.5, which is higher than that seen in prior studies with the JCTSE scoring scale based on video assessment [11]. Because of subjective variability and other factors that impact rater reliability, summative feedback, such as that used in pass-fail evaluations in the clinical setting, may be similarly less useful at this point in skills assessment in the simulation laboratory.…”

“…The assessment tool, which is based on a modified Objective Structured Assessment of Technical Skills (OSATS), included 12 component skills scored on a 5-point Likert scale (1 ¼ poor; 5 ¼ excellent), with descriptive anchors specific to each component [2,5,9,11]. Also, a pass-fail grade was given based on overall performance in executing the coronary anastomosis, as judged by the rater.…”

“…A concern for simulation-based learning is the resource and time intensiveness of attending surgeons required to teach residents in the laboratory and to assess using direct observation or a video-based modality. Regarding the latter, some have utilized edits to shorten video recordings before rating [8,11]; in this study, we chose not to edit the video recordings other than to preserve the anonymity of participants. In addition, because of the small sample size, our study may not be adequately powered, and type II error is a possibility.…”

“…Several studies have documented resident skills improvement on coronary anastomosis and mitral valve simulators at a single institution [9,10]. Recorded video has become an important adjunct to assessment based on direct observation, and interrater reliability of video-based grading has been established among experienced CT surgeons [11]. We aimed to provide CT programs and trainees a heightened awareness of simulation by offering a low-technology simulator and a healthy competition to hone their skills under faculty mentorship.…”

Training less-experienced faculty improves reliability of skills assessment in cardiac surgery

“…[3][4][5][6] For the JCTSE coronary artery anastomosis assessment tool, high inter-rater reliability among experienced educators and senior faculty members, even without rater training, has been demonstrated. 7 Because junior faculty members with less experience as educators are often charged with evaluating trainee competence, it is requisite that they achieve similar levels of inter-rater reliability.…”