Deep Placental Vessel Segmentation for Fetoscopic Mosaicking

Bano, Sophia; Vasconcelos, Francisco; Shepherd, Luke M.; Poorten, Emmanuel Vander; Vercauteren, Tom; Ourselin, Sébastien; David, Anna L.; Deprest, Jan; Stoyanov, Danail

doi:10.1007/978-3-030-59716-0_73

Cited by 30 publications

(90 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this setting, most breakthroughs have been achieved with direct methods. This includes pixel-wise gradient alignment [18], a deep learning approach for direct homography regression [19], [20], and more recently, registration of segmented placental vessels [13]. While this last method shows significant progress, it ignores accumulative drift errors that would eventually occur in long videos.…”

Section: Related Workmentioning

confidence: 99%

“…Uncalibrated camera projections are commonly modelled as a projective transformation P (3). In in-vivo fetoscopy however, due to uncalibrated distortions from the lens and water refraction [28], approximating it with an affine approximation renders iterative estimation more stable as indicated in [13], [18]. Therefore we aimed to derive an algorithm that works on 2D affine space to optimise the mosaicking directly without camera calibration.…”

Section: Related Workmentioning

confidence: 99%

“…Odometry is estimated with the method presented in [13]. We briefly introduce it in this section to make this letter self-contained.…”

Section: A the Front-end Odometrymentioning

confidence: 99%

“…al. [13], that performs vessel segmentation followed by registration of vessel probability maps. A keyframe library is then established based on sparse sampling of visual odometry frames.…”

mentioning

confidence: 99%

“…3. We test our complete mosaicking algorithm on real fetoscopic sequences, showing its advantage compared to the baseline [13] without back-end optimisation. Additionally, the affine graph-optimisation formulation is further validated with simulation data.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Globally Optimal Fetoscopic Mosaicking Based on Pose Graph Optimisation With Affine Constraints

Bano

Deprest

et al. 2021

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

Fetoscopic laser ablation surgery could be guided using a high-quality panorama of the operating site, representing a map of the placental vasculature. This can be achieved during the initial inspection phase of the procedure using image mosaicking techniques. Due to the lack of camera calibration in the operating room, it has been mostly modelled as an affine registration problem. While previous work mostly focuses on image feature extraction for visual odometry, the challenges related to large-scale reconstruction (re-localisation, loop closure, drift correction) remain largely unaddressed in this context. This letter proposes using pose graph optimisation to produce globally optimal image mosaics of placental vessels. Our approach follows the SLAM framework with a front-end for visual odometry and a back-end for long-term refinement. Our front-end uses a recent state-of-the-art odometry approach based on vessel segmentation, which is then managed by a key-frame structure and the bagof-words (BoW) scheme to retrieve loop closures. The backend, which is our key contribution, models odometry and loop closure constraints as a pose graph with affine warpings between states. This problem in the special Euclidean space cannot be solved by existing pose graph algorithms and available libraries such as G2O. We model states on affine Lie group with local linearisation in its Lie algebra. The cost function is established using Mahalanobis distance with the vectorisation of the Lie algebra. Finally, an iterative optimisation algorithm is adopted to minimise the cost function. The proposed pose graph optimisation is first validated on simulation data with a synthetic trajectory that has different levels of noise and different numbers of loop closures. Then the whole system is validated using real fetoscopic data that has three sequences with different numbers of frames and loop closures. Experimental results validate the advantage of the proposed method compared with baselines.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

“…Odometry is estimated with the method presented in [13]. We briefly introduce it in this section to make this letter self-contained.…”

Section: A the Front-end Odometrymentioning

confidence: 99%

“…al. [13], that performs vessel segmentation followed by registration of vessel probability maps. A keyframe library is then established based on sparse sampling of visual odometry frames.…”

mentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Globally Optimal Fetoscopic Mosaicking Based on Pose Graph Optimisation With Affine Constraints

Bano

Deprest

et al. 2021

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

show abstract

Autonomous Magnetic Navigation in Endoscopic Image Mosaics

Mattille,

Boehler,

Lussi

et al. 2024

Advanced Science

View full text Add to dashboard Cite

Endoscopes navigate within the human body to observe anatomical structures with minimal invasiveness. A major shortcoming of their use is their narrow field‐of‐view during navigation in large, hollow anatomical regions. Mosaics of endoscopic images can provide surgeons with a map of the tool's environment. This would facilitate procedures, improve their efficiency, and potentially generate better patient outcomes. The emergence of magnetically steered endoscopes opens the way to safer procedures and creates an opportunity to provide robotic assistance both in the generation of the mosaic map and in navigation within this map. This paper proposes methods to autonomously navigate magnetic endoscopes to 1) generate endoscopic image mosaics and 2) use these mosaics as user interfaces to navigate throughout the explored area. These are the first strategies, which allow autonomous magnetic navigation in large, hollow organs during minimally invasive surgeries. The feasibility of these methods is demonstrated experimentally both in vitro and ex vivo in the context of the treatment of twin‐to‐twin transfusion syndrome. This minimally invasive procedure is performed in utero and necessitates coagulating shared vessels of twin fetuses on the placenta. A mosaic of the vasculature in combination with autonomous navigation has the potential to significantly facilitate this challenging surgery.

show abstract

Deep learning‐based survival prediction of brain tumor patients using attention‐guided 3D convolutional neural network with radiomics approach from multimodality magnetic resonance imaging

Mazher,

Qayyum,

Puig

et al. 2023

Int J Imaging Syst Tech

View full text Add to dashboard Cite

Automatic survival prediction of gliomas from brain magnetic resonance imaging (MRI) volumes is an essential step for a patient's prognosis analysis. Radiomics research delivers beneficial feature information from MRI imaging which is substantially required by clinicians and oncologists for predicting disease prognosis for precise surgical treatment and planning. In recent years, the success of deep learning has been vast in the field of medical imaging, and it shows state‐of‐the‐art performance in applications like segmentation, classification, regression, and detection. Therefore, in this paper, we proposed a collective method using deep learning and radiomics techniques for the survival prediction of brain tumor patients. We first propose a hierarchical channel attention (HAM) module and a multi‐scale‐aware feature enhancement (MSAFE) to efficiently fuse adjacent hierarchical features in the proposed segmentation model. After segmentation, deep/latent features (LCNN) are extracted from the bottom layer of the proposed segmentation model. Later, we extracted selected radiomics features (histogram, location, and shape) using input images and segmented masks from the proposed segmentation model. Further, the 3D deep learning regressor has been trained for 3D regressor‐based deep feature extraction. We proposed the method of overall survival prediction for the brain tumor patients by combining all the meaningful features including clinical features (age) that also favorably contribute to the survival days prediction for the glioma's patients. To predict the survival days for each patient, the selected features are trained to analyze the performance of various regression techniques like random forest (RF), decision tree (DT), and XGBoost. Our proposed combined feature‐based method achieved the highest performance for survival days prediction over the state‐of‐the‐art methods. We also perform extensive experiments to show the effectiveness of each feature extraction method. The experimental results infer that deep learning‐based features along with radiomic features and clinical features are truly vital paradigms to estimate survival days.

show abstract

Deep Placental Vessel Segmentation for Fetoscopic Mosaicking

Cited by 30 publications

References 18 publications

Globally Optimal Fetoscopic Mosaicking Based on Pose Graph Optimisation With Affine Constraints

Globally Optimal Fetoscopic Mosaicking Based on Pose Graph Optimisation With Affine Constraints

Autonomous Magnetic Navigation in Endoscopic Image Mosaics

Deep learning‐based survival prediction of brain tumor patients using attention‐guided 3D convolutional neural network with radiomics approach from multimodality magnetic resonance imaging

Contact Info

Product

Resources

About