Reproducible Naevus Counts Using 3D Total Body Photography and Convolutional Neural Networks

Abstract

<b><i>Background:</i></b> The number of naevi on a person is the strongest risk factor for melanoma; however, naevus counting is highly variable due to lack of consistent methodology and lack of inter-rater agreement. Machine learning has been shown to be a valuable tool for image classification in dermatology. <b><i>Objectives:</i></b> To test whether automated, reproducible naevus counts are possible through the combination of convolutional neural networks (CNN) and three-dimensional (3D) total body imaging. <b><i>Methods:</i></b> Total body images from a study of naevi in the general population were used for the training (82 subjects, 57,742 lesions) and testing (10 subjects; 4,868 lesions) datasets for the development of a CNN. Lesions were labelled as naevi, or not (“non-naevi”), by a senior dermatologist as the gold standard. Performance of the CNN was assessed using sensitivity, specificity, and Cohen’s kappa, and evaluated at the lesion level and person level. <b><i>Results:</i></b> Lesion-level analysis comparing the automated counts to the gold standard showed a sensitivity and specificity of 79% (76–83%) and 91% (90–92%), respectively, for lesions ≥2 mm, and 84% (75–91%) and 91% (88–94%) for lesions ≥5 mm. Cohen’s kappa was 0.56 (0.53–0.59) indicating moderate agreement for naevi ≥2 mm, and substantial agreement (0.72, 0.63–0.80) for naevi ≥5 mm. For the 10 individuals in the test set, person-level agreement was assessed as categories with 70% agreement between the automated and gold standard counts. Agreement was lower in subjects with numerous seborrhoeic keratoses. <b><i>Conclusion:</i></b> Automated naevus counts with reasonable agreement to those of an expert clinician are possible through the combination of 3D total body photography and CNNs. Such an algorithm may provide a faster, reproducible method over the traditional in person total body naevus counts.

The Future of Precision Prevention for Advanced Melanoma

“…In addition naevus counts are time-consuming and therefore studies often rely on self-report, which tends to have low agreement with experts, and can lead to misclassification of risk ( 25 ). As part of the deep image-based phenotype, automated objective naevus counts can be obtained using convolutional neural networks applied to 3D total body photography ( 26 ).…”

“…Additionally, such techniques are being applied to clinical images to identify suspicious naevi ( 48 ). Image processing methods are also being used by software such as Canfield Scientific Inc (Parsippany, NJ, USA) VAM module, which can identify individual lesions from 3D total body photography, and provide lesion metrics such as diameter, hue and asymmetry ( 26 ). Additionally, through image processing and markerless tracking technology, lesions can be tracked over time to monitor changes in color, size and shape.…”

“…In addition naevus counts are time-consuming and therefore studies often rely on self-report, which tends to have low agreement with experts, and can lead to misclassification of risk (25). As part of the deep image-based phenotype, automated objective naevus counts can be obtained using convolutional neural networks applied to 3D total body photography (26). UV exposure is the primary environmental risk factor, but quantifying an individual's chronic exposure level has been difficult, largely relying on self-report as to time in the sun and protective strategies used.…”

Segmentation-Consistent Probabilistic Lesion Counting

“…Though simple to deploy, the non-differentiability of this counting approach makes it unsuitable for gradient-based training. Naturally, several parametric models have also been proposed recently: e.g., deep count regression model (Dubost et al, 2020;Han et al, 2021), patch-wise classification model (Betz-Stablein et al, 2021). However, while differentiable, these models are inherently less interpretable and less robust due to their highly-parametrized nature.…”

Monitoring of Pigmented Skin Lesions Using 3D Whole Body Imaging

“…Although there exist numerous recent articles that demonstrated the benefits of 3D whole body photography [5,24,36], 3D human representation is obtained with expensive commercial systems such as VECTRA®WB360 (Canfield Scientific Inc) [13], which simultaneously captures 92 images, using them to reconstruct a 3D avatar. In contrast, we propose a low-cost mobile health pod equipped with AI based smart tools that can be repurposed from current clinician use case, to help researchers collect and curate data automatically from 3D human models as illustrated in Fig.…”