Augmenting Multi-Instance Multilabel Learning with Sparse Bayesian Models for Skin Biopsy Image Analysis

Abstract

Skin biopsy images can reveal causes and severity of many skin diseases, which is a significant complement for skin surface inspection. Automatic annotation of skin biopsy image is an important problem for increasing efficiency and reducing the subjectiveness in diagnosis. However it is challenging particularly when there exists indirect relationship between annotation terms and local regions of a biopsy image, as well as local structures with different textures. In this paper, a novel method based on a recent proposed machine learning model, named multi-instance multilabel (MIML), is proposed to model the potential knowledge and experience of doctors on skin biopsy image annotation. We first show that the problem of skin biopsy image annotation can naturally be expressed as a MIML problem and then propose an image representation method that can capture both region structure and texture features, and a sparse Bayesian MIML algorithm which can produce probabilities indicating the confidence of annotation. The proposed algorithm framework is evaluated on a real clinical dataset containing 12,700 skin biopsy images. The results show that it is effective and prominent.

Multi‐instance multi‐label learning for surgical image annotation

“…Although this value was set empirically, it provided a reasonable tradeoff between oversegmentation and undersegmentation of the images. A similar segmentation setting has been applied in related works . In the future, we aim to perform quantitative analysis about the impact of this parameter in the annotation outcome and also to compare different segmentation algorithms.…”

“…A similar segmentation setting has been applied in related works. 19,31 In the future, we aim to perform quantitative analysis about the impact of this parameter in the annotation outcome and also to compare differ- Although under this assumption the relationship between instances is ignored, there are MIL algorithms (eg, mi-Graph 50 ) that take into account the structure of the bag. In the future, we aim to combine our method with such MIL algorithms and compare them with the outcome of MLL algorithms.…”

“…A challenging step is to select the appropriate regions from which the features will be extracted, something that is quite application dependent. For example, in Mercan et al, the features were extracted from rectangular patches, whereas in other studies the representation is based on visual words of feature descriptors extracted after keypoint detection or after image segmentation into multiple regions . The latter approach seems more suitable in surgical images where the illumination is not uniform, especially from the center of camera projection to the periphery of the image plane.…”

“…In the medical domain, MIML algorithms have been proposed for various applications, as described in a recent review . For example, in Zhang et al, a sparse Bayesian MIML algorithm was proposed for the semantic annotation of histological images of the skin. The images were multilabeled from a set of 15 annotation terms, and the instances were represented by feature vectors extracted from local regions.…”

Artificial Intelligence Applications in Dermatology: Where Do We Stand?

“…Applications have been described that can automate histology image processing and classification (67)(68)(69)(70)(71). For example, Arevalo et al described a system that analyzes histopathological images and can classify basal cell carcinoma with 98.1% accuracy (67).…”