A Holistic Multimedia System for Gastrointestinal Tract Disease Detection

Pogorelov, Konstantin; Eskeland, Sigrun Losada; Lange, Thomas de; Griwodz, Carsten; Randel, Kristin Ranheim; Stensland, Håkon Kvale; Dang-Nguyen, Duc-Tien; Spampinato, Concetto; Johansen, Dag; Riegler, Michael; Halvorsen, Pål

doi:10.1145/3083187.3083189

Cited by 67 publications

(104 citation statements)

References 50 publications

Supporting

Mentioning

103

Contrasting

Order By: Relevance

“…For the FDSI sub-task, we have also opted for a CNN and a transfer learning-based classification approach, previously validated in a different application domain in our work [41]. In fact, we initially tried to apply the well-performing GAN approach introduced in our previous works for the flood detection satellite imagery [2] and medical imagery [42,43].…”

Section: Methodology For Fdsi Taskmentioning

confidence: 99%

Automatic detection of passable roads after floods in remote sensed and social media data

Ahmad

Pogorelov

Riegler

et al. 2019

Signal Processing: Image Communication

Self Cite

View full text Add to dashboard Cite

This paper addresses the problem of floods classification and floods aftermath detection utilizing both social media and satellite imagery. Automatic detection of disasters such as floods is still a very challenging task. The focus lies on identifying passable routes or roads during floods. Two novel solutions are presented, which were developed for two corresponding tasks at the MediaEval 2018 benchmarking challenge. The tasks are (i) identification of images providing evidence for road passability and (ii) differentiation and detection of passable and non-passable roads in images from two complementary sources of information. For the first challenge, we mainly rely on object and scene-level features extracted through multiple deep models pre-trained on the ImageNet and Places datasets. The object and scene-level features are then combined using early, late and double fusion techniques. To identify whether or not it is possible for a vehicle to pass a road in satellite images, we rely on Convolutional Neural Networks and a transfer learning-based classification approach. The evaluation of the proposed methods are carried out on the largescale datasets provided for the benchmark competition. The results demonstrate significant improvement in the performance over the recent state-of-art approaches.

show abstract

Section: Methodology For Fdsi Taskmentioning

confidence: 99%

Automatic detection of passable roads after floods in remote sensed and social media data

Ahmad

Pogorelov

Riegler

et al. 2019

Signal Processing: Image Communication

Self Cite

View full text Add to dashboard Cite

show abstract

“…As a natural step forward, we designed and implemented deep-learning-based [117] and deep-featurebased [87] single-and multi-class classifiers for the detection subsystem, and evaluated and compared them with global-feature-based classifiers [99]. We demonstrated that our detection system can reach a detection performance comparable with state-of-the-art polyp detection approaches, while providing higher processing speeds and reaching our real-time goals [91,119].…”

Section: Discussionmentioning

confidence: 99%

“…Furthermore, they can also tear intestinal walls in case of severe medical conditions, and endoscopies such as enteroscopy and push enteroscopy are uncomfortable for the patients. They are performed in real-time and are challenging to scale to a larger population [91]. Also, the procedure is expensive.…”

Section: Gastrointestinal Tract Case Studymentioning

confidence: 99%

Lire

Lux

Riegler

Halvorsen

et al. 2016

Proceedings of the 7th International Conference on Multimedia Systems

Self Cite

View full text Add to dashboard Cite

Advanced and automated medical systems have been in the research focus for a long time. Together with the rapid development of sensing devices, the modern information analysis methods allow the new wave of computer-assisted systems to improve health care, quality of life, and patient survival rate. Together with the traditional computer vision and medical imaging, core competencies of the multimedia community such as integration and analysis of data from several sources, real-time processing and the assessment of usefulness for end-users play an essential role for the successful improvement of health care systems addressing challenges and open problems in the field of medicine. Our work explores different fields in multimedia research, starting from collection and annotation of multimedia data through automatic analysis of content and efficient processing of workloads to visualization and results representation. We have researched and developed a holistic medical multimedia system addressing a use case with an important medical and societal impact. We target lesions and findings detection and localization in the gastrointestinal (GI) tract of the human body in order to be able to support medical experts in their daily routine work. The early and precise detection of abnormalities in the GI tract greatly increases the chance of successful treatment if the initial observation of disease indicators occurs before the patient notices any symptoms, it is a non-trivial task that can be, however, efficiently automated. We investigated the GI tract visual analysis from a multimedia research point of view via several steps of research and development. First, we looked into the problem of medical data acquisition. We collected, annotated, and published several datasets and data annotation tools as open source. Then, we designed and developed a set of lesion and findings detection and localization approaches based on hand-crafted methods as well as on global-, local-and deepfeature-based methods, which serves as the algorithmic basis of our system. Next, we created a holistic medical multimedia system called DeepEIR. We researched and developed different subsystems for our DeepEIR system, namely (i) the data exploration and annotation subsystem, which makes it possible to collect and annotate data and transfer knowledge from medical experts into our system; (ii) the detection and localization subsystem, which perform medical data analysis in order to detect and localize lesions and findings; and (iii) the visualization and results representation subsystem that provides the information to medical personnel. Furthermore, the focus of the DeepEIR system lies on the accurate and time-efficient processing of multimedia data. We investigated, therefore, parallel and distributed processing, GPU-based acceleration and different classification and segmentation approaches that are evaluated and compared with state-of-the-art methods, algorithms, and systems. We demonstrated that the DeepEIR system could outperform state-of-the-art app...

show abstract

“…A machine learning approach that has been reborn and lately gained a lot of interest is neural networks [14] which is a type of machine learning which loosely mimics how a biological brain learns, i.e., being able to learn general concepts from concrete examples. Neural networks are also used in the medical domain, e.g., for micro-calcification detection in mammogram images [15], detecting breast cancer [3], colonic polyp detection [5], [6], [7], [8], [9] and lung cancer [4]. To perform the analysis of the data, deep neural networks (deep learning) contain multiple network layers where each layer can learn different abstraction levels of the data using the input of previous layers.…”

Section: Background and Related Workmentioning

confidence: 99%

“…Thus, detection and removal of polyps therefore prevents the development of cancer, and our detection challenge is shown in figure 1, where the polyps must be detected and identified among images and video frames also containing normal mucosa and various other anatomical landmarks and abnormalities. With respect to using neural networks for this type of abnormality detection, research have already proven that neural networks can be suitable, e.g., for similar problems detecting breast cancer [3], lung cancer [4], and for polyp detection in particular [5], [6], [7], [8], [9]. In this paper, we enhance our neural network-based EIR system (named after a Norse goddess associated with medical skills) with data enhancement methods and assess their effect on detection performance.…”

Section: Introductionmentioning

confidence: 99%

Performance of Data Enhancements and Training Optimization for Neural Network: A Polyp Detection Case Study

Henriksen

Jensen

Stensland

et al. 2019

2019 IEEE 32nd International Symposium on Computer-Based Medical Systems (CBMS)

Self Cite

View full text Add to dashboard Cite

Deep learning using neural networks is becoming more and more popular. It is frequently used in areas like video analysis, image retrieval, traffic forecast and speech recognition. In this respect, the learning and training process usually requires a lot of data. However, in many areas, data is scarce which is definitely the case in our medical application scenario, i.e., polyp detection in the gastrointestinal tract. Here, colorectal cancer is on the list of most common cancer types, and often, the cancer arises from benign, adenomatous polyps containing dysplastic cells. Detection and removal of polyps can therefore prevent the development of cancer. Due to high cost, time consumption, patient discomfort and inaccuracy of existing procedures, researchers have started to explore systems for automatic polyp detection to assist and automate current examination procedures. Following the current gained traction for neural networks, and the typical lack of medical data, we explore how data enhancements affect the training and evaluation of the networks in terms of polyp detection accuracy and particularly if it can be used to increase the detection rate. We also experiment with how various training techniques can be used to increase performance. Our experimental results show how data enhancement and training optimization can be used to increase different aspects of the performance, but we also point out mechanisms that have no and even a negative effect.

show abstract

A Holistic Multimedia System for Gastrointestinal Tract Disease Detection

Cited by 67 publications

References 50 publications

Automatic detection of passable roads after floods in remote sensed and social media data

Automatic detection of passable roads after floods in remote sensed and social media data

Lire

Performance of Data Enhancements and Training Optimization for Neural Network: A Polyp Detection Case Study

Contact Info

Product

Resources

About