Myeloma Spine and Bone Damage Score (MSBDS) on Whole-Body Computed Tomography (WBCT): Multiple Reader Agreement in a Multicenter Reliability Study

Abstract

Objective: To assess the reliability of the myeloma spine and bone damage score (MSBDS) across multiple readers with different levels of expertise and from different institutions. Methods: A reliability exercise, including 104 data sets of static images and complete CT examinations of patients affected by multiple myeloma (MM), was performed. A complementary imaging atlas provided detailed examples of the MSBDS scores, including low-risk and high-risk lesions. A total of 15 readers testing the MSBDS were evaluated. ICC estimates and their 95% confidence intervals were calculated based on mean rating (k = 15), absolute agreement, a two-way random-effects model and Cronbach’s alpha. Results: Overall, the ICC correlation coefficient was 0.87 (95% confidence interval: 0.79–0.92), and the Cronbach’s alpha was 0.93 (95% confidence interval: 0.94–0.97). Global inter- and intra-observer agreement among the 15 readers with scores below or equal to 6 points and scores above 6 points were 0.81 (95% C.I.: 0.72–0.86) and 0.94 (95% C.I.:0.91–0.98), respectively. Conclusion: We present a consensus-based semiquantitative scoring systems for CT in MM with a complementary CT imaging atlas including detailed examples of relevant scoring techniques. We found substantial agreement among readers with different levels of experience, thereby supporting the role of the MSBDS for possible large-scale applications. Significance and Innovations • Based on previous work and definitions of the MSBDS, we present real-life reliability data for quantitative bone damage assessment in multiple myeloma (MM) patients on CT. • In this study, reliability for the MSBDS, which was tested on 15 readers with different levels of expertise and from different institutions, was shown to be moderate to excellent. • The complementary CT imaging atlas is expected to enhance unified interpretations of the MSBDS between different professionals dealing with MM patients in their routine clinical practice.