Vision Transformer‐based recognition of diabetic retinopathy grade

Wu, Jianfang; Hu, Ruo; Xiao, Zhenghong; Chen, Jiaxu; Liu, Jingwei

doi:10.1002/mp.15312

Cited by 63 publications

(26 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Wu et al. ( 26 ) successfully employed Vision Transformer ( 27 ) to recognize diabetic retinopathy grade with more accuracy than the CNN-based model. Yang et al.…”

Section: Introductionmentioning

confidence: 99%

A more effective CT synthesizer using transformers for cone-beam CT-guided adaptive radiotherapy

et al. 2022

View full text Add to dashboard Cite

PurposeThe challenge of cone-beam computed tomography (CBCT) is its low image quality, which limits its application for adaptive radiotherapy (ART). Despite recent substantial improvement in CBCT imaging using the deep learning method, the image quality still needs to be improved for effective ART application. Spurred by the advantages of transformers, which employs multi-head attention mechanisms to capture long-range contextual relations between image pixels, we proposed a novel transformer-based network (called TransCBCT) to generate synthetic CT (sCT) from CBCT. This study aimed to further improve the accuracy and efficiency of ART.Materials and methodsIn this study, 91 patients diagnosed with prostate cancer were enrolled. We constructed a transformer-based hierarchical encoder–decoder structure with skip connection, called TransCBCT. The network also employed several convolutional layers to capture local context. The proposed TransCBCT was trained and validated on 6,144 paired CBCT/deformed CT images from 76 patients and tested on 1,026 paired images from 15 patients. The performance of the proposed TransCBCT was compared with a widely recognized style transferring deep learning method, the cycle-consistent adversarial network (CycleGAN). We evaluated the image quality and clinical value (application in auto-segmentation and dose calculation) for ART need.ResultsTransCBCT had superior performance in generating sCT from CBCT. The mean absolute error of TransCBCT was 28.8 ± 16.7 HU, compared to 66.5 ± 13.2 for raw CBCT, and 34.3 ± 17.3 for CycleGAN. It can preserve the structure of raw CBCT and reduce artifacts. When applied in auto-segmentation, the Dice similarity coefficients of bladder and rectum between auto-segmentation and oncologist manual contours were 0.92 and 0.84 for TransCBCT, respectively, compared to 0.90 and 0.83 for CycleGAN. When applied in dose calculation, the gamma passing rate (1%/1 mm criterion) was 97.5% ± 1.1% for TransCBCT, compared to 96.9% ± 1.8% for CycleGAN.ConclusionsThe proposed TransCBCT can effectively generate sCT for CBCT. It has the potential to improve radiotherapy accuracy.

show abstract

“…Wu et al. ( 26 ) successfully employed Vision Transformer ( 27 ) to recognize diabetic retinopathy grade with more accuracy than the CNN-based model. Yang et al.…”

Section: Introductionmentioning

confidence: 99%

A more effective CT synthesizer using transformers for cone-beam CT-guided adaptive radiotherapy

et al. 2022

View full text Add to dashboard Cite

show abstract

“…AS-OCT is, on the strength of its technical advancement, becoming an increasingly potent imaging modality for evaluation of various ocular diseases in the field of ophthalmology. In addition, the ViT, which is an extended, recent application of Transformer to computer vision inspired by its success in natural language processing, has attained excellent results in image classification and has shown its usefulness in ophthalmology as well [ 40 , 41 ]. Indeed, our pilot study demonstrated a promising potential of ViT for prediction of age from AS-OCT images.…”

Section: Discussionmentioning

confidence: 99%

Predicting demographic characteristics from anterior segment OCT images with deep learning: A study protocol

Lee

Sun²,

Kim³

2022

PLoS ONE

View full text Add to dashboard Cite

Introduction Anterior segment optical coherence tomography (AS-OCT) is a non-contact, rapid, and high-resolution in vivo modality for imaging of the eyeball’s anterior segment structures. Because progressive anterior segment deformation is a hallmark of certain eye diseases such as angle-closure glaucoma, identification of AS-OCT structural changes over time is fundamental to their diagnosis and monitoring. Detection of pathologic damage, however, relies on the ability to differentiate it from normal, age-related structural changes. Methods and analysis This proposed large-scale, retrospective cross-sectional study will determine whether demographic characteristics including age can be predicted from deep learning analysis of AS-OCT images; it will also assess the importance of specific anterior segment areas of the eyeball to the prediction. We plan to extract, from SUPREME®, a clinical data warehouse (CDW) of Seoul National University Hospital (SNUH; Seoul, South Korea), a list of patients (at least 2,000) who underwent AS-OCT imaging between 2008 and 2020. AS-OCT images as well as demographic characteristics including age, gender, height, weight and body mass index (BMI) will be collected from electronic medical records (EMRs). The dataset of horizontal AS-OCT images will be split into training (80%), validation (10%), and test (10%) datasets, and a Vision Transformer (ViT) model will be built to predict demographics. Gradient-weighted Class Activation Mapping (Grad-CAM) will be used to visualize the regions of AS-OCT images that contributed to the model’s decisions. The accuracy, sensitivity, specificity, and area under the receiver operating characteristic (ROC) curve (AUC) will be applied to evaluate the model performance. Conclusion This paper presents a study protocol for prediction of demographic characteristics from AS-OCT images of the eyeball using a deep learning model. The results of this study will aid clinicians in understanding and identifying age-related structural changes and other demographics-based structural differences. Trial registration Registration ID with open science framework: 10.17605/OSF.IO/FQ46X.

show abstract

“…Similarly, in diabetic retinopathy classification, Sun et al [101] proposed a lesion-aware Transformer architecture that jointly learns to detect the presence of diabetic retinopathy and the location of lesion discovery, using an encoder-decoder structure. Several authors have addressed the task of diabetic retinopathy recognition in a multi-class setting (i.e., no diabetic retinopathy, mild non-proliferative diabetic retinopathy, moderate diabetic retinopathy, severe non-proliferative diabetic retinopathy, and proliferative diabetic retinopathy) with a Vision Transformer architecture [102,103]. Yang et al [104] used a hybrid CNN-Transformer architecture to tackle ophthalmic image data in a multi-class setting (i.e., normal, diabetes, glaucoma, cataract, age-related macular degeneration, hypertension, myopia, and other abnormalities) using different data pre-processing strategies.…”

Section: ) Retinal Disease Classificationmentioning

confidence: 99%

A Survey on Attention Mechanisms for Medical Applications: are we Moving Toward Better Algorithms?

et al. 2022

View full text Add to dashboard Cite

The increasing popularity of attention mechanisms in deep learning algorithms for computer vision and natural language processing made these models attractive to other research domains. In healthcare, there is a strong need for tools that may improve the routines of the clinicians and the patients. Naturally, the use of attention-based algorithms for medical applications occurred smoothly. However, being healthcare a domain that depends on high-stake decisions, the scientific community must ponder if these high-performing algorithms fit the needs of medical applications. With this motto, this paper extensively reviews the use of attention mechanisms in machine learning methods (including Transformers) for several medical applications based on the types of tasks that may integrate several works pipelines of the medical domain. This work distinguishes itself from its predecessors by proposing a critical analysis of the claims and potentialities of attention mechanisms presented in the literature through an experimental case study on medical image classification with three different use cases. These experiments focus on the integrating process of attention mechanisms into established deep learning architectures, the analysis of their predictive power, and a visual assessment of their saliency maps generated by post-hoc explanation methods. This paper concludes with a critical analysis of the claims and potentialities presented in the literature about attention mechanisms and proposes future research lines in medical applications that may benefit from these frameworks.

show abstract

Vision Transformer‐based recognition of diabetic retinopathy grade

Cited by 63 publications

References 50 publications

A more effective CT synthesizer using transformers for cone-beam CT-guided adaptive radiotherapy

A more effective CT synthesizer using transformers for cone-beam CT-guided adaptive radiotherapy

Predicting demographic characteristics from anterior segment OCT images with deep learning: A study protocol

A Survey on Attention Mechanisms for Medical Applications: are we Moving Toward Better Algorithms?

Contact Info

Product

Resources

About