Comparison of inter- and intra-observer variability of breast density assessments using the fourth and fifth editions of Breast Imaging Reporting and Data System

Abstract

BackgroundBreast density is a well-known independent risk factor for breast cancer and can significantly affect the sensitivity of screening mammograms.ObjectiveWe aimed to evaluate the intra- and inter-observer consistencies of breast density assessments using methods outlined in the fourth and fifth editions of the American College of Radiology (ACR) Breast Imaging Reporting and Data System (BI-RADS) guidelines to determine which method is more reliable.Materials and methodsThree radiologists with subspecialties in breast imaging defined breast density in 72 mammograms four times each: twice using the fourth edition of the ACR BI-RADS guidelines and twice using the fifth edition. The intra- and inter-observer agreements were calculated and compared for each method.ResultsThe weighted kappa values for the overall intra-observer agreement were 0.955 (95% confidence interval [CI]: 0.931–0.980) and 0.938 (95% CI: 0.907–0.968) when breast densities were assessed according to criteria outlined in the fourth and fifth ACR BI-RADS editions, respectively. The difference between these values was not statistically significant (p = .4). The overall Fleiss-Cohen (quadratic) weighted kappa for inter-observer agreement were 0.623 (95% CI: 0.517–0.729) and 0.702 (95% CI: 0.589–0.815) when breast densities were assessed according to criteria outlined in the fourth and fifth ACR BI-RADS editions, respectively. The difference between these values was not statistically significant (p = .32). Similarly, there were no significant differences in the evaluation of breast density (overall) when comparing breast density assignment using criteria outlined in the fourth and fifth ACR BI-RADS edition (p = .582).ConclusionThe ACR BI-RADS guideline is an acceptable method to classify breast density, resulting in substantial inter-observer agreements using criteria outlined in both the fourth and fifth editions. The intra-observer agreement was nearly perfect for radiologists using criteria outlined in both sets of guidelines. Moreover, although the percentage of women who were classified as having dense breasts was higher when radiologists used the fifth edition of ACR BI-RADS guidelines than when they used the fourth edition, this difference was not statistically significant.

Comparison of breast density assessments according to BI-RADS 4th and 5th editions and experience level

“…In the literature, intra-reader agreement levels were found to be substantial and nearly perfect (18,22,23,31,32). Similar to the results reported in the literature, we found a high intra-reader agreement level, suggesting that visual methods might be reproducible without any effect from the experience level of the reader.…”

“…A study by Alikhassi et al. ( 32 ), which used a methodology similar to the present study, showed that the percentage of dense breasts was higher when radiologists used the 5th edition of the ACR BI-RADS guidelines than when they used the 4th edition. This finding was not in agreement with those reported in the literature; however, this difference was not found to be statistically significant in their study.…”

“…When using the 5th edition of the BI-RADS guidelines instead of the 4th edition, studies aimed to clarify similarly to us that more subjective judgment with leaving the percentage assessment might be done about breast density or on the other hand the simplicity by analyzing the possibility of lesions obscured by fibroglandular tissue might bring more consistency (31)(32)(33)(34)(35). In 2016, Irshad et al (33) reported lower inter-reader agreement using the 5th edition (k ¼ 0.57) than when using the 4th edition (k ¼ 0.63).…”

“…The majority of studies in the literature that analyzed the changes when using the 5th edition instead of the previous version, found statistical differences between the dense and non-dense groups; this might affect the screening approach, and different supplemental methods may come into question (33)(34)(35). A study by Alikhassi et al (32), which used a methodology similar to the present study, showed that the percentage of dense breasts was higher when radiologists used the 5th edition of the ACR BI-RADS guidelines than when they used the 4th edition. This finding was not in agreement with those reported in the literature; however, this difference was not found to be statistically significant in their study.…”

Trends in Clinical Breast Density Assessment From the Breast Cancer Surveillance Consortium

“…Our findings demonstrate a stable pattern in clinical breast density assessment in the BCSC despite the changes to density assessment guidance in the BI-RADS 5th edition. These findings stand in contrast to prior reader studies (19)(20)(21) and a single institution study (22) that suggested a small increase in dense categories with the BI-RADS 5th edition. The two prior US studies were limited to fellowship-trained breast imagers at a single institution (19,22).…”