Relevance-Based Compression of Cataract Surgery Videos Using Convolutional Neural Networks

Ghamsarian, Negin; Amirpour, Hadi; Timmerer, Christian; Taschwer, Mario; Schöffmann, Klaus

doi:10.1145/3394171.3413658

Cited by 15 publications

(9 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To evaluate the performance of our approach, we make use of data from three datasets. These include the "Cornea" [6] and "Instruments" mask annotations from the CaDIS dataset [9]. In addition, we have collected a separate dataset from which we performed the "Intraocular Lens" and "Pupil" pixel-wise segmentations 4 .…”

Section: Methodsmentioning

confidence: 99%

“…To help train future surgeons and optimize surgical workflows, automated methods that analyze cataract surgery videos have gained significant traction in the last decade. With the prospect of reducing intra-operative and post-operative complications [5], recent methods have included surgical skill assessment [8,26], remaining surgical time estimation [13], irregularity detection [7] or relevance-based compression [6]. In addition, a reliable relevant-instance-segmentation approach is often a prerequisite for a majority of these applications [17].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

DeepPyramid: Enabling Pyramid View and Deformable Pyramid Reception for Semantic Segmentation in Cataract Surgery Videos

Ghamsarian¹,

Taschwer²,

Sznitman³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Semantic segmentation in cataract surgery has a wide range of applications contributing to surgical outcome enhancement and clinical risk reduction. However, the varying issues in segmenting the different relevant structures in these surgeries make the designation of a unique network quite challenging. This paper proposes a semantic segmentation network, termed DeepPyramid, that can deal with these challenges using three novelties: (1) a Pyramid View Fusion module which provides a varying-angle global view of the surrounding region centering at each pixel position in the input convolutional feature map; (2) a Deformable Pyramid Reception module which enables a wide deformable receptive field that can adapt to geometric transformations in the object of interest; and (3) a dedicated Pyramid Loss that adaptively supervises multi-scale semantic feature maps. Combined, we show that these modules can effectively boost semantic segmentation performance, especially in the case of transparency, deformability, scalability, and blunt edges in objects. We demonstrate that our approach performs at a state-of-the-art level and outperforms a number of existing methods with a large margin (3.66% overall improvement in intersection over union compared to the best rival approach).

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

DeepPyramid: Enabling Pyramid View and Deformable Pyramid Reception for Semantic Segmentation in Cataract Surgery Videos

Ghamsarian¹,

Taschwer²,

Sznitman³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Since the seminal work by Google [10], which shows the initial success of deep CNNs for CFP-based diabetic retinopathy (DR) screening, many deep learning based methods have been proposed for eye disease recognition [2,6,8,9,13,16,24]. The majority of the methods target at a single disease, e.g., DR [8], Pathological Myopia (PM) [6], Glaucoma [24] or AMD [14], and make predictions based on a single-modal input, either a CFP or an OCT image.…”

Section: Related Workmentioning

confidence: 99%

Multi-Modal Multi-Instance Learning for Retinal Disease Recognition

Yang²,

Wang

et al. 2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

“…Being one of the most frequently performed surgeries, enhancing the outcomes of cataract surgery and diminishing its potential intra-operative and post-operative risks is of great importance. Accordingly, a large body of research has been focused on computerized surgical workflow analysis in cataract surgery [24,10,8,17,16,9], with a majority of approaches relying on semantic segmentation. Hence, improving semantic segmentation accuracy in cataract surgery videos can play a leading role in the development of a reliable computerized clinical diagnosis or surgical analysis approach [19,18].…”

Section: Introductionmentioning

confidence: 99%

ReCal-Net: Joint Region-Channel-Wise Calibrated Network for Semantic Segmentation in Cataract Surgery Videos

Ghamsarian¹,

Taschwer²,

Putzgruber-Adamitsch³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Semantic segmentation in surgical videos is a prerequisite for a broad range of applications towards improving surgical outcomes and surgical video analysis. However, semantic segmentation in surgical videos involves many challenges. In particular, in cataract surgery, various features of the relevant objects such as blunt edges, color and context variation, reflection, transparency, and motion blur pose a challenge for semantic segmentation. In this paper, we propose a novel convolutional module termed as ReCal module, which can calibrate the feature maps by employing region intra-and-inter-dependencies and channel-region cross-dependencies. This calibration strategy can effectively enhance semantic representation by correlating different representations of the same semantic label, considering a multi-angle local view centering around each pixel. Thus the proposed module can deal with distant visual characteristics of unique objects as well as cross-similarities in the visual characteristics of different objects. Moreover, we propose a novel network architecture based on the proposed module termed as ReCal-Net. Experimental results confirm the superiority of ReCal-Net compared to rival state-of-the-art approaches for all relevant objects in cataract surgery. Moreover, ablation studies reveal the effectiveness of the ReCal module in boosting semantic segmentation accuracy.

show abstract

Relevance-Based Compression of Cataract Surgery Videos Using Convolutional Neural Networks

Cited by 15 publications

References 14 publications

DeepPyramid: Enabling Pyramid View and Deformable Pyramid Reception for Semantic Segmentation in Cataract Surgery Videos

DeepPyramid: Enabling Pyramid View and Deformable Pyramid Reception for Semantic Segmentation in Cataract Surgery Videos

Multi-Modal Multi-Instance Learning for Retinal Disease Recognition

ReCal-Net: Joint Region-Channel-Wise Calibrated Network for Semantic Segmentation in Cataract Surgery Videos

Contact Info

Product

Resources

About