A self-learning teacher-student framework for gastrointestinal image classification

Abstract

Medical data is growing at an estimated 2.5 exabytes per year [1]. However, medical data is often sparse and unavailable for the research community, and qualified medical personnel rarely have time for the tedious labeling work required to prepare the data. New screening methods of the gastrointestinal (GI) tract, like video capsule endoscopy (VCE), can help to reduce patients discomfort and help to increase screening capabilities. One of the main reasons why VCE is not more commonly used by medical experts is the amount of data it produces. A high level of extra work is required by the physicians who, depending on the patient, have to look at more than 50,000 frames per examination. To make VCE more accepted and useful data analysis methods such as machine learning can be very useful.Even if a lot of frames are collected per patient they are most of the time showing normal tissue without any relevant finding. This introduces another problem, namely that it is difficult to train a machine learning based method using this data. Existing models often struggle with the challenge of not having enough data that contains anomalies. This often leads to overfitted and not generalisable models. Our work explores ways to help existing models to overcome this problem by utilising a popular sub-category of machine learning called semi-supervised learning. Semi-supervised learning uses a combination of labeled and unlabeled data which allows us to take advantage of large amounts of unlabeled data.In this thesis, we introduce our proposed semi-supervised teacher-student framework. This framework is built specifically to take advantage of vast amount of unlabeled data and consists of three main steps: (1) train a teacher model with labeled data, (2) use the teacher model to infer pseudo labels with unlabeled data, and (3) train a new and larger student model with a combination of labeled images and inferred pseudo labels. These three steps are repeated several times by treating the student as a teacher to relabel the unlabeled data and consequently training a new student.We demonstrate that our framework can be of use for classifying both, VCE and endoscopic colonoscopy images or videos. We demonstrate that our teacher-student model can significantly increase the performance compared to traditional supervisedlearning-based models. We believe that our framework has potential to be a useful addition to existing medical multimedia systems for automatic disease detection, because new data can be continuously added to improve the models performance while in production.i ii I would like to thank my supervisors Pål Halvorsen and Michael Riegler for all the help and motivation which has kept me going throughout my thesis, and for the opportunity of working on this research topic. I also wish to express my gratitude towards the two PhD student; Steven Hicks and Vajira Thambawita. Thank you for all your help, advice and support, of which I was blessed with during late nights, weekends, holidays and a global pandemic. You are the wisest m...

DeepSynthBody: the beginning of the end for data deficiency in medicine

“…To achieve the main objective, seven datasets [23,24,25,26,27,28,29], 12 benchmark analysis studies and ML models to use with CAD systems [30,31,38,39,40,68,41,36,32,33,35,34] and eight GAN studies [72,73,77,74,70,67,75,76,71] were published to cover all the sub-objectives and finally achieve the main objective and answer the research question. Some of these papers contribute to multiple objectives, while others contribute to only a single objective.…”

“…In the gastroenterology branch, classification models [30,31,32,33,34] to classify GI-tract findings and segmentation models [35,36] to segment polyp regions were investigated. When producing these ML solutions, we identified that generalizability is one of the main issues for both classification and segmentation due to the lack of labeled and annotated data to train ML models.…”

“…In this regard, this thesis discuss baseline experiments and benchmark experiments. The baseline experiments are discussed with our dataset papers [23,24,25,26,27,28,29], and the benchmark experiments are discussed in our benchmark articles [30,38,39,40,68,41,36,32,33,35,34].…”

“…Therefore, contributions of ML methods with corresponding evaluations are presented in our series of benchmark articles [30,38,39,40,68,41,36,32,33,35,34] in addition to the evaluations presented in our dataset publications [23,24,25,26,27,28,29].…”

A Novel Multi-Feature Fusion Method for Classification of Gastrointestinal Diseases Using Endoscopy Images

“…This network utilised double autoencoders for the segmentation of polyps. Gjestang et al proposed a teacher-student framework for the classification of GI diseases [ 30 ]. This network utilises unlabelled data for better generalisation.…”