Contrastive Transformer-Based Multiple Instance Learning for Weakly Supervised Polyp Frame Detection

Tian, Yu; Pang, Guansong; Liu, Fengbei; Liu, Yuyuan; Wang, Chong; Chen, Yuanhong; Verjans, Johan; Carneiro, Gustavo

doi:10.1007/978-3-031-16437-8_9

Cited by 15 publications

(6 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [9], the MIL method proposed incorporates a convolutional transform-based MIL anomaly classifier for the detection of polyps in colonoscopy videos. This method is particularly notable for its ability to operate with weaklylabelled videos, dividing each video into chunks.…”

Section: Resultsmentioning

confidence: 99%

“…In analysing the articles that mention the use of transformers [6], [9], [18], [24], [25], [26], we observed various implementations and significant impacts in the field of medical image analysis. In the article [6], the CWC-Transformer model was developed for WSI classification, combining contrastive learning with CNNs and transformers.…”

Section: B Transformersmentioning

confidence: 99%

“…This approach stands out for its ability to capture both local and global information from the images, providing a richer and more detailed analysis. In [9], a MIL method based on transformers is presented for the detection of polyps in colonoscopy videos with the ability to operate with weakly labelled videos and capture global temporal relationships, surpassing previous techniques and demonstrating the effectiveness of transformers in video contexts. In [18], the Trans-MIL model uses Neystrom's Method [49] to process many instances in high-resolution images, an important innovation to…”

Section: B Transformersmentioning

confidence: 99%

“…PatchCamelyon [36] LGViT [24] 91.8% -BreakHis [31] GMIL [17] 96.4% -99.8% 99.7% -98.6% [37] TGA-MIL [5] 92.7% 98.3% Tcga-Lung [38] CWC Transformer [6] 92.6% 94.7%/ NSCLC [39]/ ILD [40] NSK [7] -59.0% -89.0%/ 59.0% Pneumonia CT DSMIRL [8] 93.0% 96.7% HyperKvasir [41] & LDPolypVideo [42] CSM [9] -98.4%…”

Section: Datasetmentioning

confidence: 99%

See 3 more Smart Citations

Multiple Instance Learning in Medical Images: A Systematic Review

Barbosa,

Ferreira,

Junior

et al. 2024

IEEE Access

View full text Add to dashboard Cite

This article presents a systematic review of Multiple Instance Learning (MIL) applied to image classification, specifically highlighting its applications in medical imaging. Motivated by the need for a comprehensive and up-to-date analysis due to the scarcity of recent reviews, this study uses defined selection criteria to systematically assess the quality and synthesize data from relevant studies. Focusing on MIL, a subfield of machine learning that deals with learning from sets of instances or "bags", this review is crucial for medical diagnosis, where accurate lesion detection is a challenge. The review details the methodologies, advances and practical implementations of MIL, emphasizing the attention-grabbing and transformative mechanisms that improve the analysis of medical images. Challenges such as the need for extensive annotated datasets and significant computational resources are discussed. In addition, the review covers three main topics: the characterization of MIL algorithms in various imaging domains, a detailed evaluation of performance metrics, and a critical analysis of data structures and computational resources. Despite these challenges, MIL offers a promising direction for research with significant implications for medical diagnostics, highlighting the importance of continued exploration and improvement in this area.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: B Transformersmentioning

confidence: 99%

Section: B Transformersmentioning

confidence: 99%

Section: Datasetmentioning

confidence: 99%

See 2 more Smart Citations

Multiple Instance Learning in Medical Images: A Systematic Review

Barbosa,

Ferreira,

Junior

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Case I: Endora as a Temporal Data Augmenter. We explore the case of using generated videos as the unlabeled instances for semi-supervised training (by FixMatch [56]) on the video-based disease diagnosis benchmark (PolyDiag [62]). Specially, we use the randomly selected n l = 40 videos in training set of Poly-Diag as labeled data, and n u = 200 generated videos as the unlabeled data in Colonoscopic [40]and CholecTriplet [43], respectively.…”

Section: Further Empirical Studiesmentioning

confidence: 99%

Identity of university Chinese heritage language learners in Hong Kong

Li¹,

李蓁²

View full text Add to dashboard Cite

The core of out-of-distribution (OOD) detection is to learn the in-distribution (ID) representation, which is distinguishable from OOD samples. Previous work applied recognition-based methods to learn the ID features, which tend to learn shortcuts instead of comprehensive representations. In this work, we find surprisingly that simply using reconstruction-based methods could boost the performance of OOD detection significantly. We deeply explore the main contributors of OOD detection and find that reconstructionbased pretext tasks have the potential to provide a generally applicable and efficacious prior, which benefits the model in learning intrinsic data distributions of the ID dataset. Specifically, we take Masked Image Modeling as a pretext task for our OOD detection framework (MOOD). Without bells and whistles, MOOD outperforms previous SOTA of one-class OOD detection by 5.7%, multi-class OOD detection by 3.0%, and near-distribution OOD detection by 2.1%. It even defeats the 10-shot-per-class outlier exposure OOD detection, although we do not include any OOD samples for our detection.

show abstract