Surgical biomicroscopy-guided intra-operative optical coherence tomography (iOCT) image super-resolution

Abstract

Purpose Intra-retinal delivery of novel sight-restoring therapies will require the precision of robotic systems accompanied by excellent visualisation of retinal layers. Intra-operative Optical Coherence Tomography (iOCT) provides cross-sectional retinal images in real time but at the cost of image quality that is insufficient for intra-retinal therapy delivery.This paper proposes a super-resolution methodology that improves iOCT image quality leveraging spatiotemporal consistency of incoming iOCT video streams. Methods To overcome the absence of ground truth high-resolution (HR) images, we first generate HR iOCT images by fusing spatially aligned iOCT video frames. Then, we automatically assess the quality of the HR images on key retinal layers using a deep semantic segmentation model. Finally, we use image-to-image translation models (Pix2Pix and CycleGAN) to enhance the quality of LR images via quality transfer from the estimated HR domain. Results Our proposed methodology generates iOCT images of improved quality according to both full-reference and no-reference metrics. A qualitative study with expert clinicians also confirms the improvement in the delineation of pertinent layers and in the reduction of artefacts. Furthermore, our approach outperforms conventional denoising filters and the learning-based state-of-the-art. Conclusions The results indicate that the learning-based methods using the estimated, through our pipeline, HR domain can be used to enhance the iOCT image quality. Therefore, the proposed method can computationally augment the capabilities of iOCT imaging helping this modality support the vitreoretinal surgical interventions of the future.

Intra-operative OCT (iOCT) Super Resolution: A Two-Stage Methodology Leveraging High Quality Pre-operative OCT Scans

“…In our study, we propose a Super-resolution pipeline for iOCT images acquired during vitreoretinal surgeries using pre-operatively acquired OCT images as HR Fig. 4: From top to bottom: LR pseudo iOCT images, SR using [17], SR using our proposed method, HR preOCT images. domain.…”

“…As ground truth HR images do not Fig. 3: From top to bottom: LR iOCT images, SR using [17], SR using our proposed method. exist, we use five different no-reference IQA metrics, described in 2.4: FID, KID, ℓ f eat , |∆GCF| and |∆FNE|.…”

“…3) and is closer to preOCT images. Our method is compared to the real iOCT images, the state-of-the-art iOCT SR techniques [16,17] and the SR using Cycle-GAN with unpaired LR and HR datasets (UnCycGAN). Regarding perceptual metrics, our method exhibit the best FID value and close to the best KID and ℓ f eat values, which demonstrates that our methods can generate SR images that are perceptually more similar to the HR domain.…”

“…However, few works have focused on intra-operative OCT image quality enhancement. In [16] iOCT quality was improved using iOCT 3D cubes as the high resolution domain, while in [17] super-resolution achieved through surgical biomicroscopy guidance.…”

“…This work concerns self-supervised super-resolution 4 of iOCT images transferring the quality from high-resolution (HR) pre-operative OCT images to lowresolution (LR) iOCT images. As access to aligned LR-HR pairs is not avail- able, previous approach [17] focused on estimating the HR image of each LR by fusing multiple aligned iOCT video frames and then performing paired superresolution. Given the fact that their estimated HR images are still of inferior quality compared to preOCT, we propose here a two-stage methodology for the task of unpaired image quality enhancement of iOCT using available preOCT images as HR domain.…”