Epitomized Summarization of Wireless Capsule Endoscopic Videos for Efficient Visualization

Abstract

A video recording of an examination by Wireless Capsule Endoscopy (WCE) may typically contain more than 55,000 video frames, which makes the manual visual screening by an experienced gastroenterologist a highly time-consuming task. In this paper, we propose a novel method of epitomized summarization of WCE videos for efficient visualization to a gastroenterologist. For each short sequence of a WCE video, an epitomized frame is generated. New constraints are introduced into the epitome formulation to achieve the necessary visual quality for manual examination, and an EM algorithm for learning the epitome is derived. First, the local context weights are introduced to generate the epitomized frame. The epitomized frame preserves the appearance of all the input patches from the frames of the short sequence. Furthermore, by introducing spatial distributions for semantic interpretation of image patches in our epitome formulation, we show that it also provides a framework to facilitate the semantic description of visual features to generate organized visual summarization of WCE video, where the patches in different positions correspond to different semantic information. Our experiments on real WCE videos show that, using epitomized summarization, the number of frames have to be examined by the gastroenterologist can be reduced to less than one-tenth of the original frames in the video.

Scalable gastroscopic video summarization via similar-inhibition dictionary selection

“…In comparison, the endoscope video of our problem is more subjective and difficult for summarization. To our best knowledge, the state-of-the-art works for endoscope video summarization merely adopt wireless capsule endoscopy (WCE) data, such as [42][43][44], while ours focuses on the summarization of the traditional gastroscopic video. Although a gastroscopic video is obtained in a supervised condition and lasts approximately 20 min, it is also necessary to select some key frames to abstract the gastroscopic video.…”

“…First, the gastroscopic video contents are diverse, and there is much redundant information, which confirms the necessity to obtain a quick idea of what happens in the video. Then, when the prolonged video, e.g., WCE, needs to be segmented into shots for processing [45,43], the gastroscopic video is long enough in terms of the information. Finally, the gastroscopic video contents are uncontrollable, although the video is obtained under the supervision of clinicians, which is similar to the video summarization of sports/news videos recorded by photographers [46][47][48].…”

Deep sparse feature selection for computer aided endoscopy diagnosis

“…Depending on the specific lesions, the diagnosis methods can be classified to handle bleeding [2], cancer [19,17], Celiac disease, Helicobacter pylori [7], polyps [20,14] and ulcers [4], motility assessment [21], tumors [6,7], Barrett's esophagus, Crohn's disease [9,18], and just classify the region into normal and abnormal [22]. Some other applications include detecting informative frames [3], WCE color video segmentation [23], summarization [24] and clustering [25]. In this paper, we intend to detect various esophagopathy and gastropathy abnormalities using traditional gastroscope.…”

Multi-Class Latent Concept Pooling for Computer-Aided Endoscopy Diagnosis

“…Over the past few years, many computer-aided endoscopy diagnosis systems have been proposed, such as feature extraction [Coimbra and Cunha 2006;Wu et al 2007;Riaz et al 2012], feature selection [Cong et al 2015;Huang et al 2008;Li and Meng 2012], lesion classification [Buchner et al 2010;Li and Meng 2009;Kumar et al 2012;Yuan et al 2015;Mamonov et al 2014], video summarization [Chu et al 2010;Mehmood et al 2014;Iakovidis et al 2010], image enhancement [Muto et al 2011;Shahidi et al 2003;Gono et al 2004], and video segmentation [Mackiewicz et al 2008;Shen et al 2012]. Even though most of the existing systems obtain attractive performance, they stand on top of large detailed annotated images.…”