Sensorless Freehand 3D Ultrasound Reconstruction via Deep Contextual Learning

Guo, Hengtao; Xu, Sheng; Wood, Bradford J.; Yan, Pingkun

doi:10.1007/978-3-030-59716-0_44

Cited by 39 publications

(23 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Considering this threshold as a target accuracy, the accuracies ranging between 5.7 and 16.4 mm obtained by our method are sufficient as an initialisation for refinement with other registration techniques. To use this initialisation reliably without a tracker, a LUS volume could then be estimated either by separately registering multiple images in time, or by using a single image registration and a freehand US compounding method [34], [35]. Another option would be to estimate the LUS probe position through laparoscopic video based tracking [36], [37].…”

Section: Discussionmentioning

confidence: 99%

Registration of Untracked 2D Laparoscopic Ultrasound to CT Images of the Liver Using Multi-Labelled Content-Based Image Retrieval

Ramalhinho

Tregidgo

Gurusamy

et al. 2021

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

Laparoscopic Ultrasound (LUS) is recommended as a standard-of-care when performing laparoscopic liver resections as it images sub-surface structures such as tumours and major vessels. Given that LUS probes are difficult to handle and some tumours are iso-echoic, registration of LUS images to a pre-operative CT has been proposed as an image-guidance method. This registration problem is particularly challenging due to the small field of view of LUS, and usually depends on both a manual initialisation and tracking to compose a volume, hindering clinical translation. In this paper, we extend a proposed registration approach using Content-Based Image Retrieval (CBIR), removing the requirement for tracking or manual initialisation. Pre-operatively, a set of possible LUS planes is simulated from CT and a descriptor generated for each image. Then, a Bayesian framework is employed to estimate the most likely sequence of CT simulations that matches a series of LUS images. We extend our CBIR formulation to use multiple labelled objects and constrain the registration by separating liver vessels into portal vein and hepatic vein branches. The value of this new labeled approach is demonstrated in retrospective data from 5 patients. Results show that, by including a series of 5 untracked images in time, a single LUS image can be registered with accuracies ranging from 5.7 to 16.4 mm with a success rate of 78%. Initialisation of the LUS to CT registration with the proposed framework could potentially enable the clinical translation of these image fusion techniques.

show abstract

Section: Discussionmentioning

confidence: 99%

Registration of Untracked 2D Laparoscopic Ultrasound to CT Images of the Liver Using Multi-Labelled Content-Based Image Retrieval

Ramalhinho

Tregidgo

Gurusamy

et al. 2021

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

show abstract

“…In the training phase, the ConvLSTM predicts relative transform parameters Θ between all adjacent frames in the skill sequence. Its loss function includes two items, the first item is the mean absolute error (MAE) loss (i.e., L1 normalization), and the second item is the case-wise correlation loss from [3], which is beneficial to improve the generalization performance.…”

Section: Loss Functionmentioning

confidence: 99%

“…Quantitative and Qualitative Analysis. The current commonly used evaluation indicator is the final drift [14,3], which is the drift of final frame of a sequence, and the drift is the distance between the center points of the frame according to the real relative position and the estimated relative position. On this basis, a series of indicators are used to evaluate the accuracy of our proposed framework in estimating the relative transform parameters among adjacent frames.…”

Section: Loss Functionmentioning

confidence: 99%

“…Table 1 & 2 summarize the overall comparison of the proposed online learning framework with other existing methods and ablation frameworks on simulated DDH or fetus complex skill sequences, respectively. "CNN" [14] and "DCL-Net" [3] are considered in this study for comparison. "SSL" and "ADL" are our proposed online self-supervised learning and adversarial learning method, respectively.…”

Section: Loss Functionmentioning

confidence: 99%

“…Prevost et al [15] pioneered the deep learning based estimation of relative motion between US images, and later they extended their works by introducing the extra optical flow [2] and integrating a sensor source [14]. Guo et al [3] proposed a deep contextual learning network for reconstruction, which utilizes 3D convolutions over US video clips for feature extraction. However, these methods mainly handle the basic cases, rather than the complex scan sequences, such as the loop scan and fast-slow scan as shown in Fig.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Self Context and Shape Prior for Sensorless Freehand 3D Ultrasound Reconstruction

Luo

Yang

Huang

et al. 2021

Preprint

View full text Add to dashboard Cite

3D ultrasound (US) is widely used for its rich diagnostic information. However, it is criticized for its limited field of view. 3D freehand US reconstruction is promising in addressing the problem by providing broad range and freeform scan. The existing deep learning based methods only focus on the basic cases of skill sequences, and the model relies on the training data heavily. The sequences in real clinical practice are a mix of diverse skills and have complex scanning paths. Besides, deep models should adapt themselves to the testing cases with prior knowledge for better robustness, rather than only fit to the training cases. In this paper, we propose a novel approach to sensorless freehand 3D US reconstruction considering the complex skill sequences. Our contribution is three-fold. First, we advance a novel online learning framework by designing a differentiable reconstruction algorithm. It realizes an end-to-end optimization from section sequences to the reconstructed volume. Second, a self-supervised learning method is developed to explore the context information that reconstructed by the testing data itself, promoting the perception of the model. Third, inspired by the effectiveness of shape prior, we also introduce adversarial training to strengthen the learning of anatomical shape prior in the reconstructed volume. By mining the context and structural cues of the testing data, our online learning methods can drive the model to handle complex skill sequences. Experimental results on developmental dysplasia of the hip US and fetal US datasets show that, our proposed method can outperform the start-of-the-art methods regarding the shift errors and path similarities.

show abstract

End-to-end Ultrasound Frame to Volume Registration

Guo

et al. 2021

Medical Image Computing and Computer Assisted Intervention – MICCAI 2021

View full text Add to dashboard Cite

Sensorless Freehand 3D Ultrasound Reconstruction via Deep Contextual Learning

Cited by 39 publications

References 15 publications

Registration of Untracked 2D Laparoscopic Ultrasound to CT Images of the Liver Using Multi-Labelled Content-Based Image Retrieval

Registration of Untracked 2D Laparoscopic Ultrasound to CT Images of the Liver Using Multi-Labelled Content-Based Image Retrieval

Self Context and Shape Prior for Sensorless Freehand 3D Ultrasound Reconstruction

End-to-end Ultrasound Frame to Volume Registration

Contact Info

Product

Resources

About