A transformer-based multi-features fusion model for prediction of conversion in mild cognitive impairment

Zheng, Guowei; Zhang, Yu; Zhao, Ziyang; Wang, Yin; Liu, Xia; Shang, Yingying; Cong, Zhaoyang; Dimitriadis, Stavros I.; Yao, Zhijun; Hu, Bin

doi:10.1016/j.ymeth.2022.04.015

Cited by 16 publications

(2 citation statements)

References 60 publications

(63 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although the evaluation metrics used differed considerably across image analysis tasks and studies making direct comparisons challenging (Table II), there was a clear performance improvement when Transf/Attention mechanisms were used across studies. Some of the studies demonstrated either large (≥ 5%) differences against the best baseline models [21,35,46,79,101,108,117,121,122,126,127,135], or moderate (<5%) but consistent improvements across different metrics evaluated [13,18,39,53,54,56,57,62,70,78,91,94,105] and/ or data used [98,100,103,105,108]. In the following paragraphs, we detail studies that followed our 2 objective generalisation criteria (see Methods): whether a model was a) trained on large data (>2,000 images, Table I) and/ or b) analysed data from heterogeneous modalities, and/ or multiple modalities and/ or multiple organ areas and/ or multiple datasets of the same modality and organ.…”

Section: Downstream Tasks and Clinical Applicationsmentioning

confidence: 99%

Is Attention all You Need in Medical Image Analysis? A Review

Papanastasiou,

Dikaios,

Huang

et al. 2024

IEEE J. Biomed. Health Inform.

View full text Add to dashboard Cite

Medical imaging is a key component in clinical diagnosis, treatment planning and clinical trial design, accounting for almost 90% of all healthcare data. CNNs achieved performance gains in medical image analysis (MIA) over the last years. CNNs can efficiently model local pixel interactions and be trained on small-scale MI data. Despite their important advances, typical CNN have relatively limited capabilities in modelling "global" pixel interactions, which restricts their generalisation ability to understand out-ofdistribution data with different "global" information. The recent progress of Artificial Intelligence gave rise to Transformers, which can learn global relationships from data. However, full Transformer models need to be trained on large-scale data and involve tremendous computational complexity. Attention and Transformer compartments ("Transf/Attention") which can well maintain properties for modelling global relationships, have been proposed as lighter alternatives of full Transformers. Recently, there is an increasing trend to co-pollinate complementary local-global properties from CNN and Transf/Attention architectures, which led to a new era of hybrid models. The past years have witnessed substantial growth in hybrid CNN-Transf/Attention models across diverse MIA problems. In this systematic review, we survey existing hybrid CNN-Transf/Attention models, review and unravel key architectural designs, analyse breakthroughs, and evaluate current and future opportunities as well as challenges. We also introduced an analysis framework on generalisation opportunities of scientific and clinical impact, based on which new data-driven domain generalisation and adaptation methods can be stimulated.

show abstract

Section: Downstream Tasks and Clinical Applicationsmentioning

confidence: 99%

Is Attention all You Need in Medical Image Analysis? A Review

Papanastasiou,

Dikaios,

Huang

et al. 2024

IEEE J. Biomed. Health Inform.

View full text Add to dashboard Cite

show abstract

“…In the mild cognitive impairment (MCI) conversion prediction field, most previous studies suffer from overfitting issues and ignore interpretability issues in medical practice. Zheng et al [12] propose a transformer-based prediction model, which fuses cortical features containing rich ROI level information to alleviate the overfitting issues and introduces occlusion analysis to improve the model interpretability. This method can aid in the clinical prediction of MCI conversion and can assess the impact of different brain regions on model decisions.…”

mentioning

confidence: 99%