In this study assessing the IOR of MSRSGC categories, fair agreement and slight agreement were found for the AUS and SUMP categories, respectively. Observers preferentially used the AUS or benign neoplasm category for SUMP cases, perhaps because of unfamiliarity with SUMP as a diagnostic option. The initial adoption of a new reporting system will require a quality assessment to ensure that the system is reliable and useful for clinicians. Cancer Cytopathol 2018;126:390-6. © 2018 American Cancer Society.