<scp>MIST</scp>: multiple instance learning network based on Swin Transformer for whole slide image classification of colorectal adenomas

Cai, Hongbin; Feng, Xiaobing; Yin, Ruomeng; Zhao, Youcai; Guo, Lingchuan; Fan, Xiangshan; Liao, Jun

doi:10.1002/path.6027

Cited by 23 publications

(22 citation statements)

References 47 publications

(69 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [26], a new method called MIST (Multiple Instance Learning for Whole Slide Image Classification of Colorectal Adenomas) was proposed to classify colorectal adenomas in whole slide images. MIST is based on the Swin Transformer for feature extraction, employing a three-stage process.…”

Section: Resultsmentioning

confidence: 99%

Multiple Instance Learning in Medical Images: A Systematic Review

Barbosa,

Ferreira,

Junior

et al. 2024

IEEE Access

View full text Add to dashboard Cite

This article presents a systematic review of Multiple Instance Learning (MIL) applied to image classification, specifically highlighting its applications in medical imaging. Motivated by the need for a comprehensive and up-to-date analysis due to the scarcity of recent reviews, this study uses defined selection criteria to systematically assess the quality and synthesize data from relevant studies. Focusing on MIL, a subfield of machine learning that deals with learning from sets of instances or "bags", this review is crucial for medical diagnosis, where accurate lesion detection is a challenge. The review details the methodologies, advances and practical implementations of MIL, emphasizing the attention-grabbing and transformative mechanisms that improve the analysis of medical images. Challenges such as the need for extensive annotated datasets and significant computational resources are discussed. In addition, the review covers three main topics: the characterization of MIL algorithms in various imaging domains, a detailed evaluation of performance metrics, and a critical analysis of data structures and computational resources. Despite these challenges, MIL offers a promising direction for research with significant implications for medical diagnostics, highlighting the importance of continued exploration and improvement in this area.

show abstract

Section: Resultsmentioning

confidence: 99%

Multiple Instance Learning in Medical Images: A Systematic Review

Barbosa,

Ferreira,

Junior

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Transformer combined with a multi-resolution strategy to learn global affine registration. Subsequent studies [20][21][22][23] have further used the Swin Transformer and its variants as a backbone network to achieve excellent results in a variety of machine vision tasks.…”

Section: Proposed a Learning-based Affine Image Registration Algorith...mentioning

confidence: 99%

“…proposed a learning‐based affine image registration algorithm that naturally utilises the Convolutional Visual Transformer combined with a multi‐resolution strategy to learn global affine registration. Subsequent studies 20–23 have further used the Swin Transformer and its variants as a backbone network to achieve excellent results in a variety of machine vision tasks.…”

Section: Related Workmentioning

confidence: 99%

Transformer‐based 2D/3D medical image registration for X‐ray to CT via anatomical features

Qu,

Zhang,

Shi

et al. 2023

Robotics Computer Surgery

View full text Add to dashboard Cite

Background2D/3D medical image registration is one of the key technologies for surgical navigation systems to perform pose estimation and achieve accurate positioning, which still remains challenging. The purpose of this study is to introduce a new method for X‐ray to CT 2D/3D registration and conduct a feasibility study.MethodsIn this study, a 2D/3D affine registration method based on feature point detection is investigated. It combines the morphological and edge features of spinal images to accurately extract feature points from the images, and uses graph neural networks to aggregate anatomical features of different points to increase the local detail information. Meanwhile, global and positional information are extracted by the Swin Transformer.ResultsThe results indicate that the proposed method has shown improvements in both accuracy and success ratio compared with other methods. The mean target registration error value reached up to 0.31 mm; meanwhile, the runtime overhead was much lower, achieving an average runtime of about 0.6 s. This ultimately improves the registration accuracy and efficiency, demonstrating the effectiveness of the proposed method.ConclusionsThe proposed method can provide more comprehensive image information and shows good prospects for pose estimation and achieving accurate positioning in surgical navigation systems.

show abstract

“…Existing methods [12], [20]- [22], [34]- [36] usually first use WSI patches to pretrain an instance-level feature extractor through self-supervised learning and then perform model training using the extracted features. Most of them [12], [20], [21], [35], [36] use contrastive self-supervised learning methods [37]- [39] to extract instance features, but this process is completely unsupervised and it can only attempt to separate all instances as much as possible instead of effectively separating positive and negative instances. In the The red box and blue box represent positive and negative instances, respectively.…”

Section: Contrastive Learning For Wsi Classificationmentioning

confidence: 99%

“…Effective Representation Learning of INS. Currently, many studies [12], [20]- [22], [34], [35] use pre-trained feature extractors to extract instance features for subsequent training, among which ImageNet pretrained method [22] and contrastive self-supervised methods [12], [20], [21], [35], [36] are most commonly used. We compared these feature extraction methods with INS on the Camelyon16 Dataset.…”

Section: B Further Analysismentioning

confidence: 99%

DGMIL: Distribution Guided Multiple Instance Learning for Whole Slide Image Classification

Luo

Liu

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Weakly supervised whole slide image classification is usually formulated as a multiple instance learning (MIL) problem, where each slide is treated as a bag, and the patches cut out of it are treated as instances. Existing methods either train an instance classifier through pseudolabeling or aggregate instance features into a bag feature through attention mechanisms and then train a bag classifier, where the attention scores can be used for instancelevel classification. However, the pseudo instance labels constructed by the former usually contain a lot of noise, and the attention scores constructed by the latter are not accurate enough, both of which affect their performance. In this paper, we propose an instance-level MIL framework based on contrastive learning and prototype learning to effectively accomplish both instance classification and bag classification tasks. To this end, we propose an instancelevel weakly supervised contrastive learning algorithm for the first time under the MIL setting to effectively learn instance feature representation. We also propose an accurate pseudo label generation method through prototype learning. We then develop a joint training strategy for weakly supervised contrastive learning, prototype learning, and instance classifier training. Extensive experiments and visualizations on four datasets demonstrate the powerful performance of our method. Codes will be available.

show abstract

MIST: multiple instance learning network based on Swin Transformer for whole slide image classification of colorectal adenomas

Cited by 23 publications

References 47 publications

Multiple Instance Learning in Medical Images: A Systematic Review

Multiple Instance Learning in Medical Images: A Systematic Review

Transformer‐based 2D/3D medical image registration for X‐ray to CT via anatomical features

DGMIL: Distribution Guided Multiple Instance Learning for Whole Slide Image Classification

Contact Info

Product

Resources

About