Assessing Wire Navigation Performance in the Operating Room

Abstract

Objective-There are no widely accepted, objective, and reliable tools for measuring surgical skill in the operating room (OR). Ubiquitous video and imaging technology provide opportunities to develop metrics that meet this need. Hip fracture surgery is a promising area in which to develop these measures because hip fractures are common, the surgery is used as a milestone for residents, and it demands technical skill. The study objective is to develop meaningful, objective measures of wire navigation performance in the OR.Design-Resident surgeons wore a head-mounted video camera while performing surgical open reduction and internal fixation using a dynamic hip screw. Data collected from video included: duration of wire navigation, number of fluoroscopic images, and the degree of intervention by the surgeon's supervisor. To determine reliability of these measurements, four independent raters performed them for two cases. Raters independently measured the tip-apex distance (TAD), which reflects the accuracy of the surgical placement of the wire, on all 7 cases. Setting-University of Iowa Hospitals and Clinics in Iowa City, IA -a public tertiary academic center.Participants-Seven surgeries were performed by seven different orthopaedic residents. All ten raters were biomedical engineering graduate students.Results-The means and standard deviations for AP, lateral, and combined TAD measurements of the 10 raters were 2.7, 1.9, and 3.7 mm, respectively, and inter-rater reliability produced a Cronbach's Alpha of 0.97. The inter-rater reliability analysis for all 9 video-based measures produced a Cronbach's Alpha of 0.99.Conclusions-Several video-based metrics were consistent across the four video reviewers and are likely to be useful for performance assessment. The TAD measurement was less reliable than previous reports have suggested but remains a valuable metric of performance. Non-experts can reliably measure these values and they offer an objective assessment of OR performance.

Developing an objective assessment of surgical performance from operating room video and surgical imagery

“…The TAD was measured on the final anterior-posterior and lateral fluoroscopic images of the wire placement using OsiriX ® software (OsiriX Foundation; Geneva, Switzerland) (Johnson et al, 2008). In a prior investigation of the reliability of our measurement methods, we found that the inter-rater reliability of the TAD was 0.97, and that of the video metrics (duration, number of images, and supervisory intervention score) was 0.99 (Taylor et al, 2016). The TAD measure was excluded for three cases where the supervising surgeon took control of the drill to place the guide wire in the final position.…”

“…A publication in Journal of Surgical Education (Taylor et al, 2016) introduced the grading methodology and assessed the reliability of the technique presented. The reliability assessment includes the TAD measure for seven cases and video metrics for two cases made by multiple graders; these cases are included in the fifteen presented here.…”

CORR Synthesis: How Have Film Review and Motion Analysis Been Used to Enhance Orthopaedic Surgical Performance?

“…After full-text review, we included 27 studies investigating the use of film review in orthopaedics including five randomized controlled trials [8, 9, 21, 36, 47] and 22 observational studies [3, 11, 12, 15, 20, 23, 24, 30, 41, 44, 50-52, 56, 61, 64, 65, 69, 77, 78, 81, 82]. We included 28 studies investigating the use of motion analysis in orthopaedics, including four randomized controlled trials [40, 58, 62, 70] and 24 observational studies [2, 7, 13, 14, 17, 18, 32, 38, 39, 43, 49, 57, 63, 67, 68, 71-73, 75, 76, 83, 84, 89, 90].…”

“…Blinded grading of surgical footage is overwhelmingly supported in previous research for its ability to differentiate among skill levels, with only one study [64] reporting difficulty differentiating above a certain skill threshold (Table 1). Surgical footage grading has been successful for basic surgical tasks such as suturing [12,77] and more advanced surgical procedures [11,20,24,41,50,51,52,65,69,78,81,82]. Surgical footage grading is also useful for tracking improvements in skill over time [3,61], and was found to be an effective method of grading arthroscopic Bankart repair skill with high inter-rater reliability [30].…”

Objective measures of operating room wire navigation performance

“…found that the inter-rater reliability of the TAD was 0.97 and of the video metrics (duration, number of images, and supervisory intervention score) was 0.99 10 . The TAD measure was excluded for three cases where the supervising surgeon took control of the drill to place the guide wire in the final position.…”