Deep-learning-based 2.5D flow field estimation for maximum intensity projections of 4D optical coherence tomography

Abstract

In microsurgery, lasers have emerged as precise tools for bone ablation. A challenge is automatic control of laser bone ablation with 4D optical coherence tomography (OCT). OCT as high resolution imaging modality provides volumetric images of tissue and foresees information of bone position and orientation (pose) as well as thickness. However, existing approaches for OCT based laser ablation control rely on external tracking systems or invasively ablated artificial landmarks for tracking the pose of the OCT probe relative to the tissue. This can be superseded by estimating the scene flow caused by relative movement between OCT-based laser ablation system and patient.Therefore, this paper deals with 2.5D scene flow estimation of volumetric OCT images for application in laser ablation. We present a semi-supervised convolutional neural network based tracking scheme for subsequent 3D OCT volumes and apply it to a realistic semi-synthetic data set of ex vivo human temporal bone specimen.The scene flow is estimated in a two-stage approach. In the first stage, 2D lateral scene flow is computed on census-transformed en-face arguments-of-maximum intensity projections. Subsequent to this, the projections are warped by predicted lateral flow and 1D depth flow is estimated. The neural network is trained semi-supervised by combining error to ground truth and the reconstruction error of warped images with assumptions of spatial flow smoothness. Quantitative evaluation reveals a mean endpoint error of (4.7 ± 3.5) voxel or (27.5 ± 20.5) µm for scene flow estimation caused by simulated relative movement between the OCT probe and bone. The scene flow estimation for 4D OCT enables its use for markerless tracking of mastoid bone structures for image guidance in general, and automated laser ablation control.

Spatio-temporal deep learning methods for motion estimation using 4D OCT image data

“…This is also supported by pair-wise processing with S-Two-Path-3D which shows a significantly higher performance than the feature stacking approach and a higher performance than Dense4D. Our proposed 4D architecture outperforms all other approaches, including the previous deep learning concepts using two volumes [9,16] and pair-wise processing. Thus, we demonstrate the effective use of full 4D spatiotemporal information with a new deep learning model.…”

“…Another approach for high-speed OCT tracking relied on phase correlation for fast motion estimation from OCT images [20]. These approaches rely on hand-crafted features which can be error-prone, and the overall motion estimation accuracy is often limited [16]. Therefore, deep learning methods have been proposed for motion estimation from OCT data.…”

“…For example, Gessert et al proposed using 3D convolutional neural networks (CNNs) for estimating a marker's pose from single 3D OCT volumes [10]. For estimating the motion between two subsequent OCT scans, Laves et al adopted a deep learning-based optical flow method [12] using 2.5D OCT projections [16]. Similarly, Gessert et al proposed a deep learning approach for motion estimation where the parameters for a motion compensation system are directly learned from 3D OCT volumes by a deep learning model [9].…”

“…Third, we also make use of temporal information at the model output by introducing a regularization strategy that forces the model to predict motion states for previous time steps within the 4D sequence. For comparison, we consider a deep learning approach using a template and a moving volume as the input [9] which is common for motion estimation [16]. In contrast to previous deep learning approaches [9,10], we do not use an additional marker and estimate motion for a tissue dataset.…”

Concept for Markerless 6D Tracking Employing Volumetric Optical Coherence Tomography

“…OCT has been studied for image registration in the contexts of tracking of artificial markers [22] and video stabilization [23]. Additionally, different deep-learning approaches have been evaluated recently for learning known marker geometries [24] or evaluating optical flow [25]. However, the constantly limiting aspect when using OCT imaging is the very small field-of-view (FOV).…”