Deep Lesion Tracker: Monitoring Lesions in 4D Longitudinal Imaging Studies

Abstract

Monitoring treatment response in longitudinal studies plays an important role in clinical practice. Accurately identifying lesions across serial imaging follow-up is the core to the monitoring procedure. Typically this incorporates both image and anatomical considerations. However, matching lesions manually is labor-intensive and time-consuming. In this work, we present deep lesion tracker (DLT), a deep learning approach that uses both appearance-and anatomical-based signals. To incorporate anatomical constraints, we propose an anatomical signal encoder, which prevents lesions being matched with visually similar but spurious regions. In addition, we present a new formulation for Siamese networks that avoids the heavy computational loads of 3D cross-correlation. To present our network with greater varieties of images, we also propose a self-supervised learning (SSL) strategy to train trackers with unpaired images, overcoming barriers to data collection. To train and evaluate our tracker, we introduce and release the first lesion tracking benchmark, consisting of 3891 lesion pairs from the public DeepLesion database. The proposed method, DLT, locates lesion centers with a mean error distance of 7mm. This is 5% better than a leading registration algorithm while running 14 times faster on whole CT volumes. We demonstrate even greater improvements over detector or similarity-learning alternatives. DLT also generalizes well on an external clinical test set of 100 longitudinal studies, achieving 88% accuracy. Finally, we plug DLT into an automatic tumor monitoring workflow where it leads to an accuracy of 85% in assessing lesion treatment responses, which is only 0.46% lower than the accuracy of manual inputs.

Weakly-Supervised Universal Lesion Segmentation with Regional Level Set Loss

“…For testing, an image is taken as input directly if its long side is in the range, otherwise, it is resized into the closest bound first. Although this work takes as input a lesion-of-interest region, it can cooperate with lesion detection and tracking techniques [25,34,33,4,32] to perform automatic lesion segmentation on the entire CT images. Quantitative Results.…”

Lesion Segmentation and RECIST Diameter Prediction via Click-driven Attention and Dual-path Connection

“…To mimic the clicking behavior of a radiologist, a point is randomly selected from a region obtained by eroding the ellipse to half of its size. Besides requiring a click, it can also cooperate with lesion detection and tracking techniques [28,36,7,8] to perform fully automatic lesion segmentation and RECIST diameter prediction.…”

“…Deep learning based computer-aided diagnosis techniques [4,5,6,7,8,9,10,11] have been extensively studied by researchers, including automatic lesion segmentation. Most existing works focused on tumors of specific types, such as lung nodules [12,13], liver tumors [14,6], and lymph nodes [15].…”