Mammographic system performance using an image reading qualification method

Abstract

Our goal was to evaluate mammography systems based on microcalcifications and fiber detection using a statistical phantom (ALVIM, model TRM 18-209, Nuclear Associates) image readings. ALVIM phantom images were acquired under diverse exposure conditions with various equipments, and 5 radiologists with similar expertise reported their findings. The reading performance in the detection of microcalcifications and fibers of different sizes was measured by simulation of equivalent breast tissue with 4.5 and 6.5 cm thicknesses. We determined kappa values, ROC curves, and kappa probability density and detection rates with dedicated software developed locally. The statistical results generated three kappa (K) ranges that allowed quantification of the detection performance at three quality levels: unacceptable (K ≤ 0.64), achievable (K ≥ 0.70) and acceptable (0.64 < K < 0.70). An extensive database permitted a comparison of the reading performance with 99.5% reliability (p < 0.005). The comparison showed a larger dispersion of the kappa values for the images with low contrast generated with mammography equipment which was not properly calibrated, showing that the method is able to detect the performance changes associated with the loss of image quality.

Physical characterization of the quality of medical images does not adequately reflect their clinical quality

“…Various other methods have been suggested 4 and, for example, a simple statistical method has been developed which defines diagnostic accuracy, sensitivity and specificity using phantoms within which the position of pathology simulators can be changed manually or with special software. [6][7][8] The method uses phantoms with several regions each containing different numbers of pathology simulators of various sizes and shapes. Within each region, the simulators are identical but spaced randomly.…”