The Influence of Surgeon Experience and Subspeciality on the Reliability of the AO Spine Sacral Classification System

Abstract

Study Design. Cross-sectional survey. Objective. To determine the influence of surgeons' level of experience and subspeciality training on the reliability, reproducibility, and accuracy of sacral fracture classification using the Arbeitsgemeinschaft fu ¨r Osteosynthesefragen Spine Sacral Classification System. Summary of Background Data. A surgeons' level of experience or subspecialty may have a significant effect on the reliability and accuracy of sacral classification given various levels of comfort with imaging assessment required for accurate diagnosis and classification. Methods. High-resolution computerized tomography (CT) images from 26 cases were assessed on two separate occasions by 172 investigators representing a diverse array of surgical subspecialities (general orthopedics, neurosurgery, orthopedic spine, orthopedic trauma) and experience (<5, 5-10, 11-20, >20 yrs). Reliability and reproducibility were calculated with Cohen kappa coefficient (k) and gold standard classification agreement was determined for each fracture morphology and subtype and stratified by experience and subspeciality. Results. Respondents achieved an overall k ¼ 0.87 for morphology and k ¼ 0.77 for subtype classification, representing excellent and substantial intraobserver reproducibility, respectively. Respondents from all four practice experience groups demonstrated excellent interobserver reliability when classifying overall morphology (k ¼ 0.842/0.850, Assessment 1/Assessment 2) and substantial interobserver reliability in overall subtype (k ¼ 0.719/ 0.751) in both assessments. General orthopedists, neurosurgeons, and orthopedic spine surgeons exhibited excellent interobserver reliability in overall morphology classification and substantial interobserver reliability in overall subtype classification. Surgeons in each experience category and subspecialty correctly classified fracture morphology in over 90% of cases and fracture subtype in over 80% of cases according to the gold standard. Correct overall classification of fracture morphology (Assessment 1: P ¼ 0.024, Assessment 2: P ¼ 0.006) and subtype (P 2 < 0.001) differed significantly by years of experience but not by subspecialty. Conclusion. Overall, the Arbeitsgemeinschaft fu ¨r Osteosynthesefragen spine sacral classification system appears to be universally applicable among surgeons of various subspecialties and levels of experience with acceptable reliability, reproducibility, and accuracy.

Classifications in Brief: The AOSpine Sacral Classification System

“…They did not find differences between spine and trauma pelvic surgeons or among surgeons with different levels of experience. Karamian et al [4] obtained better interrater and intrarater agreement using this classification. In this study, 26 patients were evaluated by 172 observers from different specialties and with different levels of expertise.…”

“…Karamian et al [4] obtained better interrater and intrarater agreement using this classification. In this study, 26 patients were evaluated by 172 observers from different specialties and with different levels of expertise.…”

“…Finally, as noted earlier, the fact that kappa values for interobserver reliability at the subtype level were generally < 0.6 [2, 12], the utility of the AOSpine sacral classification system for evaluating subtypes is limited, and we believe it is not high enough to recommend for clinical or research use at the subtype level. However, considering the recent publication by Karamian et al [4], in which the authors observed higher interobserver agreement by implementing a different training session strategy, we believe that more reliability studies are warranted.…”

“…Thus, we recommend being cautious in using this scheme at the subtype level in practice and research. Because one study about new training methods showed better interrater agreement [4], we believe that more agreement studies are warranted using approaches such as that one. Future prospective studies are required to establish whether this classification allows surgeons to build an evidence-based treatment algorithm.…”

An international validation of the AO spine subaxial injury classification system

“…Even though our results suggest there is no correlation between surgeon experience and improved AO Spine Subaxial Injury Classification System reliability or reproducibility, no study has examined if increased application of the classification to cervical spine injuries improves a participants accuracy. Of note, no previous study has found a correlation between surgeon experience and the reliability and reproducibility of different AO Spine classifications [12,13].…”