Unsupervised brain imaging 3D anomaly detection and segmentation with transformers

Pinaya, Walter Hugo Lopez; Tudosiu, Petru-Daniel; Gray, Robert; Rees, Geraint; Nachev, Parashkev; Ourselin, Sébastien; Cardoso, M. Jorge

doi:10.1016/j.media.2022.102475

Cited by 74 publications

(34 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the follow-up papers on these approaches showed that the performance translates to other medical datasets as well [40], [53], [54]. Here, in contrast to the two purely self-supervised proxy task methods, the two other top performing methods use Autoencoder-based methods, which are another main direction in anomaly detection [19], [21], [23] and follow-up and consecutive work has also extended the methods to other datasets with great success [46], [55].…”

Section: Discussionmentioning

confidence: 99%

MOOD 2020: A Public Benchmark for Out-of-Distribution Detection and Localization on Medical Images

Zabala-Blanco

Full

Isensee

et al. 2022

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

Detecting Out-of-Distribution (OoD) data is one of the greatest challenges in safe and robust deployment of machine learning algorithms in medicine. When the algorithms encounter cases that deviate from the distribution of the training data, they often produce incorrect and over-confident predictions. OoD detection algorithms aim to catch erroneous predictions in advance by analysing the data distribution and detecting potential instances of failure. Moreover, flagging OoD cases may support human readers in identifying incidental findings. Due to the increased interest in OoD algorithms, benchmarks for different domains have recently been established. In the medical imaging domain, for which reliable predictions are often essential, an open benchmark has been missing. We introduce the Medical-Out-Of-Distribution-Analysis-Challenge (MOOD) as an open, fair, and unbiased benchmark for OoD methods in the medical imaging domain. The analysis of the submitted algorithms shows that performance has a strong positive correlation with the perceived difficulty, and that all algorithms show a high variance for different anomalies, making it yet hard to recommend them for clinical practice.We also see a strong correlation between challenge ranking and performance on a simple toy test set, indicating that this might be a valuable addition as a proxy dataset during anomaly detection algorithm development.

show abstract

Section: Discussionmentioning

confidence: 99%

MOOD 2020: A Public Benchmark for Out-of-Distribution Detection and Localization on Medical Images

Zabala-Blanco

Full

Isensee

et al. 2022

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

show abstract

“…Other approaches propose restoration methods (Chen et al, 2020), uncertainty estimation (Sato et al, 2019), adversarial autoencoders (Chen and Konukoglu, 2018) or the use of encoder activation maps (Silva-Rodríguez et al, 2022). Also, vector-quantized VAEs have been proposed (Pinaya et al, 2022b). As an alternative to AE-based architectures, generative adversarial networks (GANs) have been applied to the problem of UAD (Schlegl et al, 2019).…”

Section: Recent Workmentioning

confidence: 99%

Unsupervised Anomaly Detection in 3D Brain MRI Using Deep Learning with Impured Training Data

Behrendt¹,

Bengs²,

Frederik³

et al. 2022

2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI)

View full text Add to dashboard Cite

The detection of lesions in magnetic resonance imaging (MRI)-scans of human brains remains challenging, timeconsuming and error-prone. Recently, unsupervised anomaly detection (UAD) methods have shown promising results for this task. These methods rely on training data sets that solely contain healthy samples. Compared to supervised approaches, this significantly reduces the need for an extensive amount of labeled training data. However, data labelling remains error-prone. We study how unhealthy samples within the training data affect anomaly detection performance for brain MRI-scans. For our evaluations, we consider three publicly available data sets and use autoencoders (AE) as a well-established baseline method for UAD. We systematically evaluate the effect of impured training data by injecting different quantities of unhealthy samples to our training set of healthy samples from T1-weighted MRI-scans. We evaluate a method to identify falsely labeled samples directly during training based on the reconstruction error of the AE. Our results show that training with impured data decreases the UAD performance notably even with few falsely labeled samples. By performing outlier removal directly during training based on the reconstruction-loss, we demonstrate that falsely labeled data can be detected and removed to mitigate the effect of falsely labeled data. Overall, we highlight the importance of clean data sets for UAD in brain MRI and demonstrate an approach for detecting falsely labeled data directly during training.

show abstract

“…A Transformer is a DL model that adopts the mechanism of self-attention [28], differently weighting the significance of each part of the input data. Transformers are also used for AD in many different fields such as, aerial videos from a UAV's [29], system logs [30] and brain-scan images [31]. The selfattention mechanism of Transformers is very useful for AD, making it easier for DL models to recognize irregular activities in the input.…”

Section: Introductionmentioning

confidence: 99%

A Stethoscope for Drones: Transformers-Based Methods for UAVs Acoustic Anomaly Detection

et al. 2023

View full text Add to dashboard Cite

Unmanned Aerial Vehicles and the increasing variety of their applications are raising in popularity. The growing number of UAVs, emphasizes the significance of drones' reliability and robustness. Thus, there is a need for an efficient self-observing sensing mechanism to detect real-time anomalies in drone behavior. Previous works suggested prediction models from control theory, yet, they are complex by nature and hard to implement, while Deep Learning solutions are of great utility. In this paper, we propose a real-time framework to detect anomalies in drones by analyzing the sound emitted from them. For this purpose, we construct a hybrid Deep Learning based Transformer and a Convolutional Neural Network inspired by the well-known VGG architecture. Our approach is examined over a dataset that is collected from a single microphone set located on a micro drone in real-time. Our approach achieves an F1score of 88.4% in detecting anomalies and outperforms the VGG-16 architecture. Moreover, the framework presented in this paper reduces the number of parameters of the well-known VGG-16 from 138M, into a shrunk version with 3.6M parameters only. Additionally, our real-time approach, results in a smaller number of parameters in the neural network, and yet yields high accuracy in anomaly detection in drones with an average inference time of 0.2 seconds per second. Moreover, with an earphone that weighs less than 100 grams on top of the UAV, our method is shown to be beneficial, even in extreme conditions such as a microsize dataset that is composed of three hours of flight recordings. The presented self-observing method can be implemented by simply adding a microphone to drones and transmitting the captured audio for analysis to the remote control or performing it onboard the drone using a dedicated microcontroller.

show abstract

Unsupervised brain imaging 3D anomaly detection and segmentation with transformers

Cited by 74 publications

References 28 publications

MOOD 2020: A Public Benchmark for Out-of-Distribution Detection and Localization on Medical Images

MOOD 2020: A Public Benchmark for Out-of-Distribution Detection and Localization on Medical Images

Unsupervised Anomaly Detection in 3D Brain MRI Using Deep Learning with Impured Training Data

A Stethoscope for Drones: Transformers-Based Methods for UAVs Acoustic Anomaly Detection

Contact Info

Product

Resources

About