Shifted-windows transformers for the detection of cerebral aneurysms in microsurgery

Abstract

Purpose Microsurgical Aneurysm Clipping Surgery (MACS) carries a high risk for intraoperative aneurysm rupture. Automated recognition of instances when the aneurysm is exposed in the surgical video would be a valuable reference point for neuronavigation, indicating phase transitioning and more importantly designating moments of high risk for rupture. This article introduces the MACS dataset containing 16 surgical videos with frame-level expert annotations and proposes a learning methodology for surgical scene understanding identifying video frames with the aneurysm present in the operating microscope’s field-of-view. Methods Despite the dataset imbalance (80% no presence, 20% presence) and developed without explicit annotations, we demonstrate the applicability of Transformer-based deep learning architectures (MACSSwin-T, vidMACSSwin-T) to detect the aneurysm and classify MACS frames accordingly. We evaluate the proposed models in multiple-fold cross-validation experiments with independent sets and in an unseen set of 15 images against 10 human experts (neurosurgeons). Results Average (across folds) accuracy of 80.8% (range 78.5–82.4%) and 87.1% (range 85.1–91.3%) is obtained for the image- and video-level approach, respectively, demonstrating that the models effectively learn the classification task. Qualitative evaluation of the models’ class activation maps shows these to be localized on the aneurysm’s actual location. Depending on the decision threshold, MACSWin-T achieves 66.7–86.7% accuracy in the unseen images, compared to 82% of human raters, with moderate to strong correlation. Conclusions Proposed architectures show robust performance and with an adjusted threshold promoting detection of the underrepresented (aneurysm presence) class, comparable to human expert accuracy. Our work represents the first step towards landmark detection in MACS with the aim to inform surgical teams to attend to high-risk moments, taking precautionary measures to avoid rupturing.

Understanding the brain with attention: A survey of transformers in brain sciences

“…Zhou et al [85] vidMACSSwin-T [85] 2023 the regression loss function, the Mean Absolute Error (MAE) is a common choice because of its robustness and reduction of loss shock caused by abnormal points. Owing to its simplicity, Mean Squared Error (MSE) is prone to be adopted for largescale samples.…”

“…In addition, TranSOP incorporated clinical meta-information to enhance a priori modeling for the modified Rankin Scale (mRS) calculation of stroke outcomes. Zhou et al [85] carried out tumor identification in Microsurgical Aneurysm Clipping Surgery (MACS). They developed vidMACSSwin-T for video frame classification to detect aneurysms, thereby enabling the surgical team to be aware of high-risk moments and take preventive action.…”

“…In addition, TranSOP incorporated clinical meta‐information to enhance a priori modeling for the modified Rankin Scale (mRS) calculation of stroke outcomes. Zhou et al [85] . carried out tumor identification in Microsurgical Aneurysm Clipping Surgery (MACS).…”