Fusion of Visual and Anamnestic Data for the Classification of Skin Lesions with Deep Learning

Bonechi, Simone; Bianchini, Monica; Bongini, Pietro; Ciano, Giorgio; Giacomini, Giorgia; Rosai, Riccardo; Tognetti, Linda; Rossi, Alberto; Andreini, Paolo

doi:10.1007/978-3-030-30754-7_21

Cited by 10 publications

(10 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, they tracked the lesion’s evolution by documenting whether the lesion increased in size or changed its shape. Furthermore, Bonechi et al [ 42 ] considered the presence of melanocytic cells as an additional potentially relevant feature.…”

Section: Resultsmentioning

confidence: 99%

“…Different techniques were used for continuous parameters such as the patient’s age. One-hot encoding is only possible after discretizing the continuous range, which was performed by Bonechi et al [ 42 ], who divided the age ranging from 0 to 95 in the sections of 5 years. Gessert et al [ 46 ] tested numerical against one-hot encoding and found the former to be superior.…”

Section: Resultsmentioning

confidence: 99%

“…In most studies, the authors decided to reduce the image features before concatenating them with patient data. In only 27% (3/11) of studies, the authors provided sufficient information on this point and revealed a considerable variance in the ratio of image features to patient data: 112 to 28 [ 53 ], 128 to 80 [ 42 ], and 2048 or 2×2048 to 11 [ 56 ]. However, reducing the image features should be done with care, as it is accompanied by a loss of information.…”

Section: Resultsmentioning

confidence: 99%

“…Bonechi et al [ 42 ] considered four types of patient data individually (age, sex, location, and presence of melanocytic cells) for a binary classification (malignant yes or no). Unfortunately, the analysis results have not been reported in detail, but the authors reported the parameter “presence of melanocytic cells” to be the most informative.…”

Section: Resultsmentioning

confidence: 99%

“…For instance, sex was included in 91% (10/11) of studies, but was stated to be of minimal benefit for the classification task if investigated in detail. Regarding the results of studies considering patient data besides these three factors, the results indicate that the integration of other patient data may be more promising [ 39 , 42 , 53 ]. Studies analyzing the risk factors for skin cancer so far demonstrated that patient data can be helpful in distinguishing skin lesions in binary classification tasks.…”

Section: Discussionmentioning

confidence: 99%

See 4 more Smart Citations

Integrating Patient Data Into Skin Cancer Classification Using Convolutional Neural Networks: Systematic Review

Höhn¹,

Hekler²,

Krieghoff-Henning³

et al. 2021

J Med Internet Res

View full text Add to dashboard Cite

Background Recent years have been witnessing a substantial improvement in the accuracy of skin cancer classification using convolutional neural networks (CNNs). CNNs perform on par with or better than dermatologists with respect to the classification tasks of single images. However, in clinical practice, dermatologists also use other patient data beyond the visual aspects present in a digitized image, further increasing their diagnostic accuracy. Several pilot studies have recently investigated the effects of integrating different subtypes of patient data into CNN-based skin cancer classifiers. Objective This systematic review focuses on the current research investigating the impact of merging information from image features and patient data on the performance of CNN-based skin cancer image classification. This study aims to explore the potential in this field of research by evaluating the types of patient data used, the ways in which the nonimage data are encoded and merged with the image features, and the impact of the integration on the classifier performance. Methods Google Scholar, PubMed, MEDLINE, and ScienceDirect were screened for peer-reviewed studies published in English that dealt with the integration of patient data within a CNN-based skin cancer classification. The search terms skin cancer classification, convolutional neural network(s), deep learning, lesions, melanoma, metadata, clinical information, and patient data were combined. Results A total of 11 publications fulfilled the inclusion criteria. All of them reported an overall improvement in different skin lesion classification tasks with patient data integration. The most commonly used patient data were age, sex, and lesion location. The patient data were mostly one-hot encoded. There were differences in the complexity that the encoded patient data were processed with regarding deep learning methods before and after fusing them with the image features for a combined classifier. Conclusions This study indicates the potential benefits of integrating patient data into CNN-based diagnostic algorithms. However, how exactly the individual patient data enhance classification performance, especially in the case of multiclass classification problems, is still unclear. Moreover, a substantial fraction of patient data used by dermatologists remains to be analyzed in the context of CNN-based skin cancer classification. Further exploratory analyses in this promising field may optimize patient data integration into CNN-based skin cancer diagnostics for patients’ benefits.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%