Multimodal skin lesion classification using deep learning

Yap, Jordan; Yolland, William; Tschandl, Philipp

doi:10.1111/exd.13777

Cited by 208 publications

(147 citation statements)

References 31 publications

Supporting

Mentioning

141

Contrasting

Unclassified

Order By: Relevance

“…Six out of eleven early fusion studies extracted features from medical imaging using a CNN (Table 1). Four out of the six studies that applied neural networks for feature extraction simply concatenated the extracted imaging features with clinical features for their fusion strategy [26][27][28][29] . The remaining two studies by Liu et al 30 and Nie et al 31 applied dimensionality reduction techniques before concatenating the features.…”

Section: Early Fusionmentioning

confidence: 99%

Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines

et al. 2020

View full text Add to dashboard Cite

Advancements in deep learning techniques carry the potential to make significant contributions to healthcare, particularly in fields that utilize medical imaging for diagnosis, prognosis, and treatment decisions. The current state-of-the-art deep learning models for radiology applications consider only pixel-value information without data informing clinical context. Yet in practice, pertinent and accurate non-imaging data based on the clinical history and laboratory data enable physicians to interpret imaging findings in the appropriate clinical context, leading to a higher diagnostic accuracy, informative clinical decision making, and improved patient outcomes. To achieve a similar goal using deep learning, medical imaging pixel-based models must also achieve the capability to process contextual data from electronic health records (EHR) in addition to pixel data. In this paper, we describe different data fusion techniques that can be applied to combine medical imaging with EHR, and systematically review medical data fusion literature published between 2012 and 2020. We conducted a systematic search on PubMed and Scopus for original research articles leveraging deep learning for fusion of multimodality data. In total, we screened 985 studies and extracted data from 17 papers. By means of this systematic review, we present current knowledge, summarize important results and provide implementation guidelines to serve as a reference for researchers interested in the application of multimodal fusion in medical imaging.

show abstract

Section: Early Fusionmentioning

confidence: 99%

Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Unbalanced classes. A frequent scenario in wound analysis is the uneven distribution of tissue patterns [32,33]. For instance, Nejati et al [24] evaluated a dataset of 350 wound images divided into labeled patches, where most of the labels were related to only three of seven possible classes.…”

Section: Preliminariesmentioning

confidence: 99%

A superpixel-driven deep learning approach for the analysis of dermatological wounds

Blanco¹,

Traina²,

Traina³

et al. 2020

Computer Methods and Programs in Biomedicine

View full text Add to dashboard Cite

Background. The image-based identification of distinct tissues within dermatological wounds enhances patients' care since it requires no intrusive evaluations. This manuscript presents an approach, we named QTDU , that combines deep learning models with superpixel-driven segmentation methods for assessing the quality of tissues from dermatological ulcers. Method. QTDU consists of a three-stage pipeline for the obtaining of ulcer segmentation, tissues' labeling, and wounded area quantification. We set up our approach by using a real and annotated set of dermatological ulcers for training several deep learning models to the identification of ulcered superpixels. Results. Empirical evaluations on 179,572 superpixels divided into four classes showed QTDU accurately spot wounded tissues (AUC = 0.986, sensitivity = 0.97, and specificity = 0.974) and outperformed machinelearning approaches in up to 8.2% regarding F1-Score through fine-tuning of a ResNet-based model. Last, but not least, experimental evaluations also showed QTDU correctly quantified wounded tissue areas within a 0.089 Mean Absolute Error ratio. Conclusions. Results indicate QTDU effectiveness for both tissue segmentation and wounded area quantification tasks. When compared to existing machine-learning approaches, the combination of superpixels and deep learning models outperformed the competitors within strong significant levels. can be automatically evaluated by Computer-Aided Diagnosis (CAD) tools, or even used for the searching of massive databases through content-only queries, as in Content-Based Image Retrieval (CBIR) applications. In both CAD and CBIR cases, the detection of abnormalities requires the extraction of patterns from images, while a decision-making strategy is necessary for juxtaposing new images to those in the database [4,5].Since dermatological lesions are routinely diagnosed by biopsies and surrounding skin aspects, ulcers can be computationally characterized by particular types of tissues (and their areas) within the wounded region [6,7]. For instance, Mukherjee et al.[8] proposed a five-color classification model and applied a color-based low-level extractor further labeled by a Support-Vector Machine (SVM) strategy at an 87.61% hit ratio. Such idea of concatenating feature extraction and classification is found at the core of most wound segmentation strategies, as in the study of Kavitha et al.[9] that evaluated leg ulcerations by extracting patterns based on local spectral histograms to be labeled by a Multi-Layer Perceptron (MLP) classifier with 87.05% accuracy. Analogously, Pereyra et al.[10] discussed the use of color descriptors and an Instancebased Learning (IbL) classifier with a 61.7% hit ratio, whereas Veredas et al. [11] suggested the use of texture descriptors and an MLP classifier with 84.84% accuracy.Blanco et al.[4] and Chino et al.[12] followed a slightly different premise for finding proper similarity measures and comparison criteria for dermatological wounds. Their approaches are based on a divide-andconquer stra...

show abstract

“…), with images of lesions and have shown that a classifier based on both patient metadata and dermoscopic images is more accurate than a classifier based only on dermoscopic images [32,44]. Other studies have combined dermoscopic images with macroscopic images [45] or with a sonification layer [39] to increase the accuracy of its classifier. When diagnosing unknown skin lesions, clinicians typically do not only visually inspect the skin, but they simultaneously take into account many points of clinical data, including patient demographics, laboratory tests, etc.…”

Section: Melanomamentioning

confidence: 99%

Machine Learning in Dermatology: Current Applications, Opportunities, and Limitations

Chan

Reddy

Myers

et al. 2020

Dermatol Ther (Heidelb)

165

108

View full text Add to dashboard Cite

Machine learning (ML) has the potential to improve the dermatologist's practice from diagnosis to personalized treatment. Recent advancements in access to large datasets (e.g., electronic medical records, image databases, omics), faster computing, and cheaper data storage have encouraged the development of ML algorithms with human-like intelligence in dermatology. This article is an overview of the basics of ML, current applications of ML, and potential limitations and considerations for further development of ML. We have identified five current areas of applications for ML in dermatology: (1) disease classification using clinical images; (2) disease classification using dermatopathology images; (3) assessment of skin diseases using mobile applications and personal monitoring devices; (4) facilitating large-scale epidemiology research; and (5) precision medicine. The purpose of this review is to provide a guide for dermatologists to help demystify the fundamentals of ML and its wide range of applications in order to better evaluate its potential opportunities and challenges.

show abstract

Multimodal skin lesion classification using deep learning

Cited by 208 publications

References 31 publications

Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines

Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines

A superpixel-driven deep learning approach for the analysis of dermatological wounds

Machine Learning in Dermatology: Current Applications, Opportunities, and Limitations

Contact Info

Product

Resources

About