Self Context and Shape Prior for Sensorless Freehand 3D Ultrasound Reconstruction

Luo, Mingyuan; Huang, Xiaoqiong; Huang, Yuhao; Zou, Yuxin; Hu, Xindi; Ravikumar, Nishant; Frangi, Alejandro F.; Ni, Dong

doi:10.1007/978-3-030-87231-1_20

Cited by 17 publications

(13 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 1 shows that our MoNet outperforms previous approaches on all metrics significantly (t-test, p < 0.05) for both arm and carotid scans. We note that the CNN incorporating IMU Euler angles [11] outperforms other sensorless approaches [2,7] in terms of EA metric, verifying the effectiveness of IMU integration. However, the simple combination of CNN and IMU results in the lowest performance on other metrics.…”

Section: Multi-modal Online Self-supervised Strategysupporting

confidence: 56%

“…Quantitative and Qualitative Analysis. To demonstrate the efficacy of our method, we performed the comparison with three SOTA approaches including CNN [11], DCL-Net [2], and OLF [7]. CNN used images, calculated optical flow and IMU Euler angles as input, whereas DCL-Net and OLF only used scanned images.…”

Section: Multi-modal Online Self-supervised Strategymentioning

confidence: 99%

“…Meanwhile, OLF's shape prior module was removed because muscles and vessels do not have prominent shape features. In this study, six criteria, including final drift rate (FDR), average drift rate (ADR), maximum drift (MD), sum of drift (SD), symmetric Hausdorff distance (HD), and mean error of angle (EA) are used to evaluate the performance (all criteria refer to [7] except EA). Furthermore, our ablation experiments compare Backbone (ResNet+LSTM), Backbone+IMU, and full MoNet (Backbone+IMU+Online) to validate the efficacy of IMU fusion and online learning strategy.…”

Section: Multi-modal Online Self-supervised Strategymentioning

confidence: 99%

“…It typically reconstructs the volume by calculating the relative positions of a series of US images. Recently, freehand reconstruction techniques have moved away from complex and costly external positioning systems previously used to obtain high-precision positions [12,11,2,7]. However, improving reconstruction accuracy remains difficult as these techniques only rely on images to infer relative positions.…”

Section: Introductionmentioning

confidence: 99%

“…Guo et al [2] proposed a deep contextual learning network (DCL-Net) to mine the correlation information of US video clips for 3D reconstruction. Luo et al [7] designed an online learning framework (OLF) that improves reconstruction performance by utilizing consistency constraints and shape priors. Although effective, the reconstruction performance based on deep learning still faces challenges such as difficult elevational displacement estimation and large cumulative drift.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Deep Motion Network for Freehand 3D Ultrasound Reconstruction

Luo¹,

Yang²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Freehand 3D ultrasound (US) has important clinical value due to its low cost and unrestricted field of view. Recently deep learning algorithms have removed its dependence on bulky and expensive external positioning devices. However, improving reconstruction accuracy is still hampered by difficult elevational displacement estimation and large cumulative drift. In this context, we propose a novel deep motion network (MoNet) that integrates images and a lightweight sensor known as the inertial measurement unit (IMU) from a velocity perspective to alleviate the obstacles mentioned above. Our contribution is two-fold. First, we introduce IMU acceleration for the first time to estimate elevational displacements outside the plane. We propose a temporal and multi-branch structure to mine the valuable information of low signalto-noise ratio (SNR) acceleration. Second, we propose a multi-modal online self-supervised strategy that leverages IMU information as weak labels for adaptive optimization to reduce drift errors and further ameliorate the impacts of acceleration noise. Experiments show that our proposed method achieves the superior reconstruction performance, exceeding state-of-the-art methods across the board.

show abstract

Section: Multi-modal Online Self-supervised Strategysupporting

confidence: 56%