Tumor detection under cystoscopy with transformer-augmented deep learning algorithm

Abstract

Objective. Accurate tumor detection is critical in cystoscopy to improve bladder cancer resection and decrease recurrence. Advanced deep learning algorithms hold the potential to improve the performance of standard white-light cystoscopy (WLC) in a noninvasive and cost-effective fashion. The purpose of this work is to develop a cost-effective, transformer-augmented deep learning algorithm for accurate detection of bladder tumors in WLC and to assess its performance on archived patient data. Approach. ‘CystoNet-T’, a deep learning-based bladder tumor detector, was developed with a transformer-augmented pyramidal CNN architecture to improve automated tumor detection of WLC. CystoNet-T incorporated the self-attention mechanism by attaching transformer encoder modules to the pyramidal layers of the feature pyramid network (FPN), and obtained multi-scale activation maps with global features aggregation. Features resulting from context augmentation served as the input to a region-based detector to produce tumor detection predictions. The training set was constructed by 510 WLC frames that were obtained from cystoscopy video sequences acquired from 54 patients. The test set was constructed based on 101 images obtained from WLC sequences of 13 patients. Main results. CystoNet-T was evaluated on the test set with 96.4 F1 and 91.4 AP (Average Precision). This result improved the benchmark of Faster R-CNN and YOLO by 7.3 points in F1 and 3.8 points in AP. The improvement is attributed to the strong ability of global attention of CystoNet-T and better feature learning of the pyramids architecture throughout the training. The model was found to be particularly effective in highlighting the foreground information for precise localization of the true positives while favorably avoiding false alarms Significance. We have developed a deep learning algorithm that accurately detects bladder tumors in WLC. Transformer-augmented AI framework promises to aid in clinical decision-making for improved bladder cancer diagnosis and therapeutic guidance.

U-Net-Based Assistive Identification of Bladder Cancer: A Promising Approach for Improved Diagnosis

“…In light of these limitations, the U-Net model still has room for development in bladder tumor recognition. Moving forward, the incorporation of more data types (including CT or MR imaging data and special imaging cystoscopy) and optimization of the model structure can lead to enhanced accuracy in tumor recognition [27,28], enabling the classification and refinement of different types and stages of bladder tumors. Chang TC, Nairveen Ali, Barrios W et al [27,[29][30][31] have attempted to utilize different types of CNNs for the classification of bladder tumor histology or invasiveness.…”

Developing an explainable diagnosis system utilizing deep learning model: a case study of spontaneous pneumothorax

“…Computer-aided diagnostics using Artificial Intelligence detects anomalies in medical images, providing accurate results for healthcare professionals (Chiu et al 2022a, Duan et al 2023, Jia et al 2023. To streamline processes, many hospitals are adopting intelligent diagnosis systems to boost efficiency and lessen physicians' workload, reflecting the rapid rise of artificial intelligence in various fields (Fu et al 2013, Yan et al 2020.…”