Application of neural network and computer vision technologies for image analysis of skin lesion

Abstract

Opportunity research of using neural networks and computer vision to analyze images of skin lesion and identify features of various pathologies, including oncological neoplasms. A methodology has been developed that makes it possible to evaluate the significance of combinations of color components and spaces in feature extraction using local binary patterns (LBP) and histogram of oriented gradients (HOG) computer vision technologies to extract features of skin changes binary classification of human skin lesions. Optimization of extracted feature makes it possible to more effectively solve the problem of data separability in classification. Research reveals an accessible way to classify skin lesions on a small dataset (less than 1000 images). Research is supposed to be applied to data sequences obtained using a new unique method of multispectral processing of skin lesions. In the course of the work, data from the ISIC-19 and ISIC-20 datasets were used. Samples were formed with a limit of 1000 images for training and validating the models. Additionally, a test sample of 250 images was formed. All images were reduced to 128 × 128 pixels and converted to YCrCb, BGR, Grayscale, HSV color spaces. Features were extracted for each color channel using the HOG and LBP methods. Mathematical models, including neural networks have been used for data classification. The effectiveness of features combinations by color channels and feature extraction methods was evaluated. The preprocessed images were divided into training and validation subsets in a 70/30 ratio. The accuracy, recall, precision and f1-score metrics were used to evaluate the models. The models were evaluated using stratified cross-validation and a test dataset. Optimization of model parameters was carried out based on the loss function represented by the average of cross-validation and evaluation on the validation set. In the process of research, more than 15 000 different optimizations of model parameters were executed. The most stable results on the validation dataset were achieved using ensemble of models, which were trained on a combination of features using local binary patterns (LBP) and histogram of oriented gradients (HOG) technologies. Models which used only local binary patterns technology had the best metrics values, but these models are not recommended to be used in practice without ensemble with stronger models. The results gained can be applied for usage with an ensemble of state-of-the-art convolutional and recurrent neural networks. The proposed approach is universal and applicable both for the analysis of individual images of skin neoplasms and for the analysis of their sequences obtained by the method of multispectral image processing. The technique can be applied to datasets with a limited amount of data. The results obtained will be of interest to specialists in the fields of computer vision and medical images analysis.