A Comparison of Lung Nodule Segmentation Algorithms: Methods and Results from a Multi-institutional Study

Abstract

Tumor volume estimation, as well as accurate and reproducible borders segmentation in medical images, are important in the diagnosis, staging, and assessment of response to cancer therapy. The goal of this study was to demonstrate the feasibility of a multi-institutional effort to assess the repeatability and reproducibility of nodule borders and volume estimate bias of computerized segmentation algorithms in CT images of lung cancer, and to provide results from such a study. The dataset used for this evaluation consisted of 52 tumors in 41 CT volumes (40 patient datasets and 1 dataset containing scans of 12 phantom nodules of known volume) from five collections available in The Cancer Imaging Archive. Three academic institutions developing lung nodule segmentation algorithms submitted results for three repeat runs for each of the nodules. We compared the performance of lung nodule segmentation algorithms by assessing several measurements of spatial overlap and volume measurement. Nodule sizes varied from 29 μl to 66 ml and demonstrated a diversity of shapes. Agreement in spatial overlap of segmentations was significantly higher for multiple runs of the same algorithm than between segmentations generated by different algorithms (p < 0.05) and was significantly higher on the phantom dataset compared to the other datasets (p < 0.05). Algorithms differed significantly in the bias of the measured volumes of the phantom nodules (p < 0.05) underscoring the need for assessing performance on clinical data in addition to phantoms. Algorithms that most accurately estimated nodule volumes were not the most repeatable, emphasizing the need to evaluate both their accuracy and precision. There were considerable differences between algorithms, especially in a subset of heterogeneous nodules, underscoring the recommendation that the same software be used at all time points in longitudinal studies.

Assessment of Stability and Discrimination Capacity of Radiomic Features on Apparent Diffusion Coefficient Images

“…Compared to [27], it does not need a digital biopsy, which requires a further segmentation step, although a digital biopsy takes less time to be segmented than a normal ROI. Compared to a method based on [28], it requires no segmentation algorithm, which can be difficult to design for oropharyngeal tumors. Last, the presented method allows to evaluate not only stability, but also the discriminative power of the features, which is something that, to the knowledge of the authors, was never considered before.…”

“…In [27], the stability analysis is performed by comparing radiomic features computed on the entire ROI, and on a “digital biopsy,” i.e., a small portion of the ROI that is large enough to capture the heterogeneity of the tumor. Last, comparison of radiomic features obtained with multiple initialization of a semi-automatic segmentation algorithm or with different segmentation algorithms (like in [28]) could potentially be used for stability assessment. Although these approaches strongly reduce the amount of manual work necessary for a stability analysis of the radiomic features, they cannot be used to evaluate the discriminative power.…”

Identification of optimal mother wavelets in survival prediction of lung cancer patients using wavelet decomposition‐based radiomic features

“…Dataset MS included CT images of patients with lung NSCLC (n = 30) that were selected from three datasets publicly available online within the Quantitative Imaging Network (QIN) multisite collection of Lung CT data with nodule segmentations . This dataset was used to assess the reproducibility of the radiomic features under different imaging and segmentation conditions.…”

“…Each institution performed the segmentation by using its in‐house segmentation algorithm and under three different initial conditions of the algorithm. These configurations have resulted in nine segmentations of each tumor, thereby yielding in total 270 segmentations …”

Quantitative imaging of cancer in the postgenomic era: Radio(geno)mics, deep learning, and habitats

“…44 Another study using the same data set revealed that many radiomics features, particularly those that quantify shape and margin sharpness, are quite sensitive to segmentation. A given region may contain an entire tumor, a subset of the tumor (eg, a habitat), and/or a peritumoral region thought to be involved with or affected by the tumor.…”