Less is More: Sample Selection and Label Conditioning Improve Skin Lesion Segmentation

D-LEMA: Deep Learning Ensembles from Multiple Annotations -- Application to Skin Lesion Segmentation

“…Table 1 compares the segmentation performance of our baseline models as well as the individual base models across different prediction fusion schemes using the Jaccard index. In addition, we compare the performance of our proposed method against the work by Ribeiro et al [28] where a subset of samples with small annotator disagreements are also taken into account during the training. In our based models, we follow Ribeiro et al [28] and minimize the loss function with respect to the randomly selected image annotations in each training batch.…”

“…All images are 8-bit RGB color dermoscopy images. Similar to [28], we utilized 2,223 images with more than one segmentation ground truth mask (2,094 with two, 100 with three and 36 with four and 3 with five) to train our model. We split all 2,223 images to 80% for training and 20% for validation.…”

“…All the images in the used data sets are cropped into 96 × 96 pixels and normalized using the per-channel mean and standard deviation across the entire data set. A brief description of these test data sets are provided as follows: ISIC: Ribeiro et al randomly selected a test set of 2,000 images with just one segmentation ground truth from ISIC Archive [28]. We used the exact set in our experimental evaluations for fair comparisons.…”

“…The same authors then proposed to train their segmentation model on a subset of the images, derived by filtering out all samples whose mean pairwise Cohen's kappa score was less than 0.5, thus using only those segmentations which largely agree between annotators [28].…”

“…Therefore, we seek training deep segmentation models to learn from multiple annotations as available instead of discarding some annotations. Rather than selecting a subset of images to learn from [28], we instead propose a generalized approach of annotation weighting by leveraging different groups of consistent annotations in an ensemble method towards efficiently learning from all available annotations. We also utilize uncertainty estimates [14,17] in an ensemble learning framework to improve predictive uncertainty and calibration confidence in the final prediction.…”