2D ultrasound imaging based intra-fraction respiratory motion tracking for abdominal radiation therapy using machine learning

Abstract

We have previously developed a robotic ultrasound imaging system for motion monitoring in abdominal radiation therapy. Owing to the slow speed of ultrasound image processing, our previous system could only track abdominal motions under breath-hold. To overcome this limitation, a novel 2D-based image processing method for tracking intra-fraction respiratory motion is proposed. Fifty-seven different anatomical features acquired from 27 sets of 2D ultrasound sequences were used in this study. Three 2D ultrasound sequences were acquired with the robotic ultrasound system from three healthy volunteers. The remaining datasets were provided by the 2015 MICCAI Challenge on Liver Ultrasound Tracking. All datasets were preprocessed to extract the feature point, and a patientspecific motion pattern was extracted by principal component analysis and slow feature analysis (SFA). The tracking finds the most similar frame (or indexed frame) by a k-dimensional-tree-based nearest neighbor search for estimating the tracked object location. A template image was updated dynamically through the indexed frame to perform a fast template matching (TM) within a learned smaller search region on the incoming frame. The mean tracking error between manually annotated landmarks and the location extracted from the indexed training frame is 1.80 ± 1.42 mm. Adding a fast TM procedure within a small search region reduces the mean tracking error to 1.14 ± 1.16 mm. The tracking time per frame is 15 ms, which is well below the frame acquisition time. Furthermore, the anatomical reproducibility was measured by analyzing the location's anatomical landmark relative to the probe; the position-controlled probe has better reproducibility and yields a smaller mean error across all three volunteer cases, compared to the force-controlled probe (2.69 versus 11.20 mm in the superior-inferior direction and 1.19 versus 8.21 mm in the anterior-posterior direction). Our method reduces the processing time for tracking respiratory motion significantly, which can reduce the delivery uncertainty.

The comparison of OpenCV library algorithms for tracking movements of liver tumors in ultrasound video.

“…In addition, the problem of real-time tracking is often solved by reducing the image fed to the tracking functions with the subsequent restoration of the original size [29], which is primarily caused by low performance, especially when processing HD images. This technology is not used for the experiment -the frames of the original size frames are examined since a hypothesis is put forward for testing the possibility of analyzing SD images by modern computing systems in real-time.…”

Artificial Intelligence in Quantitative Ultrasound Imaging: A Review

“…A temporal consistency model was employed as a location prior and combined with the network-predicted location probability map for tracking the target in the ultrasound sequences. Huang et al developed a DL-based real-time motion tracking technique for ultrasound image-guided radiation therapy by combining the attention-aware fully CNN (FCNN) and the convolutional long short-term memory network (CLSTM) [113,114]. FCNN extracted spatial features while CLSTM refined the features via computing the saliency mask.…”

Deep learning‐based motion tracking using ultrasound images

“…As advances in deep learning algorithms, 40–47 convolutional neural network (CNN)‐based motion tracking methods have achieved initial promising results 48–55 . Nouri et al 48 .…”

“…As advances in deep learning algorithms, [40][41][42][43][44][45][46][47] convolutional neural network (CNN)-based motion tracking methods have achieved initial promising results. [48][49][50][51][52][53][54][55] Nouri et al 48 first reported a CNN-based algorithm to track landmarks in US image sequence by learning a distance metric of US images patches. To improve the accuracy and robustness of tracking, encoderdecoder networks, 54 recurrent neural network, 55 long short-term memory (LSTM) network, 49 and Siamese networks 50 have been introduced.…”