ReCal-Net: Joint Region-Channel-Wise Calibrated Network for Semantic Segmentation in Cataract Surgery Videos

Ghamsarian, Negin; Taschwer, Mario; Putzgruber-Adamitsch, Doris; Sarny, Stephanie; El-Shabrawi, Yosuf; Schoeffmann, Klaus

doi:10.1007/978-3-030-92238-2_33

Cited by 6 publications

(3 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Attention mechanisms can be broadly described as the techniques to guide the network's computational resources (i.e.,the convolutional operations) toward the most determinative features in the input feature map [9,15,16]. Such mechanisms have been especially proven to be gainful in the case of semantic segmentation.…”

Section: Attention Modulesmentioning

confidence: 99%

DeepPyramid+: medical image segmentation using Pyramid View Fusion and Deformable Pyramid Reception

Ghamsarian,

Wolf,

Zinkernagel

et al. 2024

Int J CARS

Self Cite

View full text Add to dashboard Cite

Purpose Semantic segmentation plays a pivotal role in many applications related to medical image and video analysis. However, designing a neural network architecture for medical image and surgical video segmentation is challenging due to the diverse features of relevant classes, including heterogeneity, deformability, transparency, blunt boundaries, and various distortions. We propose a network architecture, DeepPyramid+, which addresses diverse challenges encountered in medical image and surgical video segmentation. Methods The proposed DeepPyramid+ incorporates two major modules, namely “Pyramid View Fusion” (PVF) and “Deformable Pyramid Reception” (DPR), to address the outlined challenges. PVF replicates a deduction process within the neural network, aligning with the human visual system, thereby enhancing the representation of relative information at each pixel position. Complementarily, DPR introduces shape- and scale-adaptive feature extraction techniques using dilated deformable convolutions, enhancing accuracy and robustness in handling heterogeneous classes and deformable shapes. Results Extensive experiments conducted on diverse datasets, including endometriosis videos, MRI images, OCT scans, and cataract and laparoscopy videos, demonstrate the effectiveness of DeepPyramid+ in handling various challenges such as shape and scale variation, reflection, and blur degradation. DeepPyramid+ demonstrates significant improvements in segmentation performance, achieving up to a 3.65% increase in Dice coefficient for intra-domain segmentation and up to a 17% increase in Dice coefficient for cross-domain segmentation. Conclusions DeepPyramid+ consistently outperforms state-of-the-art networks across diverse modalities considering different backbone networks, showcasing its versatility. Accordingly, DeepPyramid+ emerges as a robust and effective solution, successfully overcoming the intricate challenges associated with relevant content segmentation in medical images and surgical videos. Its consistent performance and adaptability indicate its potential to enhance precision in computerized medical image and surgical video analysis applications.

show abstract

Section: Attention Modulesmentioning

confidence: 99%

DeepPyramid+: medical image segmentation using Pyramid View Fusion and Deformable Pyramid Reception

Ghamsarian,

Wolf,

Zinkernagel

et al. 2024

Int J CARS

Self Cite

View full text Add to dashboard Cite

show abstract

“…Recent studies have also explored automated technical skill assessment in robotic surgeries [51]. Given the fundamental role of semantic segmentation in various surgical workflow analysis applications, significant efforts have been invested in improving semantic segmentation performance in cataract surgery [16], [17], [20]. In recent years, some efforts have been made to enable surgical prognosis, such as post-surgical visual acuity prediction [67].…”

Section: Related Workmentioning

confidence: 99%

Predicting Postoperative Intraocular Lens Dislocation in Cataract Surgery via Deep Learning

Ghamsarian,

Putzgruber-Adamitsch,

Sarny

et al. 2024

IEEE Access

Self Cite

View full text Add to dashboard Cite

A critical yet unpredictable complication following cataract surgery is intraocular lens dislocation. Postoperative stability is imperative, as even a tiny decentration of multifocal lenses or inadequate alignment of the torus in toric lenses due to postoperative rotation can lead to a significant drop in visual acuity. Investigating possible intraoperative indicators that can predict post-surgical instabilities of intraocular lenses can help prevent this complication. In this paper, we develop and evaluate the first fully-automatic framework for the computation of lens unfolding delay, rotation, and instability during surgery. Adopting a combination of three types of CNNs, namely recurrent, region-based, and pixelbased, the proposed framework is employed to assess the possibility of predicting post-operative lens dislocation during cataract surgery. This is achieved via performing a large-scale study on the statistical differences between the behavior of different brands of intraocular lenses and aligning the results with expert surgeons' hypotheses and observations about the lenses. We exploit a large-scale dataset of cataract surgery videos featuring four intraocular lens brands. Experimental results confirm the reliability of the proposed framework in evaluating the lens' statistics during the surgery. The Pearson correlation and ttest results reveal significant correlations between lens unfolding delay and lens rotation and significant differences between the intra-operative rotations stability of four groups of lenses. These results suggest that the proposed framework can help surgeons select the lenses based on the patient's eye conditions and predict post-surgical lens dislocation.

show abstract

“…ZhenLiang Ni et al [10] proposed an attention model to improve network features, and solve the problem of mirror reflection of surgical instruments. Ghamsarian et al [ 11 ] proposed the ReCal convolution module to segment cataracts, and use dependencies to calibrate the feature map, which can effectively correlate the information within the region and the cross-channel region information.…”

Section: Semantic Segmentation In Cataract Surgery Videosmentioning

confidence: 99%

Image Segmentation of Endoscopic Cataract Based on Semantic Segmentation

Chen

Yin

et al. 2022

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

Medical imaging technology has become one of the indispensable computer-assisted intervention methods in clinical disease diagnosis and treatment, including identifying and locating lesion areas, detecting and segmenting tissue and organ lesions in different modalities. The wide application of medical image analysis in clinical examination and medical auxiliary diagnosis has effectively improved the diagnostic efficiency of doctors. In this paper, we propose a novel and effective model to achieve semantic segmentation of cataract datasets. This model uses the self-supervised method BYOL for parameter pre-training, which improves the model’s ability to extract image consistency features. In addition, we have added a lightweight Coordinate attention mechanism to the backbone network to enable the model to independently learn the correlation between the channel and the space and enhance the ability of network feature expression. Experiments are conducted on the cataract endoscopy fine-grained segmentation data set, showing the effectiveness of the proposed method for segment the organs and surgical instruments in the cataract surgical microscope image, which demonstrates the accuracy and robustness of the proposed method.

show abstract

ReCal-Net: Joint Region-Channel-Wise Calibrated Network for Semantic Segmentation in Cataract Surgery Videos

Cited by 6 publications

References 26 publications

DeepPyramid+: medical image segmentation using Pyramid View Fusion and Deformable Pyramid Reception

DeepPyramid+: medical image segmentation using Pyramid View Fusion and Deformable Pyramid Reception

Predicting Postoperative Intraocular Lens Dislocation in Cataract Surgery via Deep Learning

Image Segmentation of Endoscopic Cataract Based on Semantic Segmentation

Contact Info

Product

Resources

About