Categorical Relation-Preserving Contrastive Knowledge Distillation for Medical Image Classification

Xing, Xiaohan; Hou, Yuenan; Li, Hang; Yuan, Yixuan; Li, Hongsheng; Meng, Max Q.-H.

doi:10.1007/978-3-030-87240-3_16

Cited by 19 publications

(6 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In their framework, transfer learning was applied to the convolutional neural network for plain and hierarchical classification and used to differentiate between seven types of skin lesions. Xing et al [ 28 ] presented a Categorical Relation-preserving Contrastive Knowledge Distillation (CRCKD) that was used as a supervisor of the model. They presented a class-guided contrastive distillation (CCD) module for closer image pairs from the same class as a teacher while separating negative images from different classes.…”

Section: Literature Reviewmentioning

confidence: 99%

An Efficient Deep Learning-Based Skin Cancer Classifier for an Imbalanced Dataset

et al. 2022

View full text Add to dashboard Cite

Efficient skin cancer detection using images is a challenging task in the healthcare domain. In today’s medical practices, skin cancer detection is a time-consuming procedure that may lead to a patient’s death in later stages. The diagnosis of skin cancer at an earlier stage is crucial for the success rate of complete cure. The efficient detection of skin cancer is a challenging task. Therefore, the numbers of skilful dermatologists around the globe are not enough to deal with today’s healthcare. The huge difference between data from various healthcare sector classes leads to data imbalance problems. Due to data imbalance issues, deep learning models are often trained on one class more than others. This study proposes a novel deep learning-based skin cancer detector using an imbalanced dataset. Data augmentation was used to balance various skin cancer classes to overcome the data imbalance. The Skin Cancer MNIST: HAM10000 dataset was employed, which consists of seven classes of skin lesions. Deep learning models are widely used in disease diagnosis through images. Deep learning-based models (AlexNet, InceptionV3, and RegNetY-320) were employed to classify skin cancer. The proposed framework was also tuned with various combinations of hyperparameters. The results show that RegNetY-320 outperformed InceptionV3 and AlexNet in terms of the accuracy, F1-score, and receiver operating characteristic (ROC) curve both on the imbalanced and balanced datasets. The performance of the proposed framework was better than that of conventional methods. The accuracy, F1-score, and ROC curve value obtained with the proposed framework were 91%, 88.1%, and 0.95, which were significantly better than those of the state-of-the-art method, which achieved 85%, 69.3%, and 0.90, respectively. Our proposed framework may assist in disease identification, which could save lives, reduce unnecessary biopsies, and reduce costs for patients, dermatologists, and healthcare professionals.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

An Efficient Deep Learning-Based Skin Cancer Classifier for an Imbalanced Dataset

et al. 2022

View full text Add to dashboard Cite

show abstract

“…On the other hand, there are also many state-of-the-art methods with great performance on skin lesion classification. The Student-and-Teacher Model is also a high-performance model introduced in 2021 [ 19 ], and it is created by Xiaohan Xing et al as a combination of two models that share memories with the other model. Therefore, the models can take full advantage of what others learn.…”

Section: Introductionmentioning

confidence: 99%

Skin Lesion Classification on Imbalanced Data Using Deep Learning with Soft Attention

Bui

Khoi

2022

Sensors

View full text Add to dashboard Cite

Today, the rapid development of industrial zones leads to an increased incidence of skin diseases because of polluted air. According to a report by the American Cancer Society, it is estimated that in 2022 there will be about 100,000 people suffering from skin cancer and more than 7600 of these people will not survive. In the context that doctors at provincial hospitals and health facilities are overloaded, doctors at lower levels lack experience, and having a tool to support doctors in the process of diagnosing skin diseases quickly and accurately is essential. Along with the strong development of artificial intelligence technologies, many solutions to support the diagnosis of skin diseases have been researched and developed. In this paper, a combination of one Deep Learning model (DenseNet, InceptionNet, ResNet, etc) with Soft-Attention, which unsupervisedly extract a heat map of main skin lesions. Furthermore, personal information including age and gender are also used. It is worth noting that a new loss function that takes into account the data imbalance is also proposed. Experimental results on data set HAM10000 show that using InceptionResNetV2 with Soft-Attention and the new loss function gives 90 percent accuracy, mean of precision, F1-score, recall, and AUC of 0.81, 0.81, 0.82, and 0.99, respectively. Besides, using MobileNetV3Large combined with Soft-Attention and the new loss function, even though the number of parameters is 11 times less and the number of hidden layers is 4 times less, it achieves an accuracy of 0.86 and 30 times faster diagnosis than InceptionResNetV2.

show abstract

“…Recently, self-supervised deep models such as contrastive learning have shown promising results in 3D medical image classification [8,19,20,1]. The pillar of contrastive learning is to augment a similar 3D image from one patient to form a "homogeneous" pair and use images from other patients to construct multiple "heterogeneous" pairs.…”

Section: Introductionmentioning

confidence: 99%

Contrastive Graph Learning for Population-based fMRI Classification

Wang¹,

Yao²,

Rekik³

2022

Preprint

View full text Add to dashboard Cite

Contrastive self-supervised learning has recently benefited fMRI classification with inductive biases. Its weak label reliance prevents overfitting on small medical datasets and tackles the high intraclass variances. Nonetheless, existing contrastive methods generate resemblant pairs only on pixel-level features of 3D medical images, while the functional connectivity that reveals critical cognitive information is under-explored. Additionally, existing methods predict labels on individual contrastive representation without recognizing neighbouring information in the patient group, whereas interpatient contrast can act as a similarity measure suitable for population-based classification. We hereby proposed contrastive functional connectivity graph learning for population-based fMRI classification. Representations on the functional connectivity graphs are "repelled" for heterogeneous patient pairs meanwhile homogeneous pairs "attract" each other. Then a dynamic population graph that strengthens the connections between similar patients is updated for classification. Experiments on a multi-site dataset ADHD200 validate the superiority of the proposed method on various metrics. We initially visualize the population relationships and exploit potential subtypes.

show abstract

Categorical Relation-Preserving Contrastive Knowledge Distillation for Medical Image Classification

Cited by 19 publications

References 23 publications

An Efficient Deep Learning-Based Skin Cancer Classifier for an Imbalanced Dataset

An Efficient Deep Learning-Based Skin Cancer Classifier for an Imbalanced Dataset

Skin Lesion Classification on Imbalanced Data Using Deep Learning with Soft Attention

Contrastive Graph Learning for Population-based fMRI Classification

Contact Info

Product

Resources

About