Recurrent attention network for false positive reduction in the detection of pulmonary nodules in thoracic CT scans

Abstract

Purpose: Multiview two-dimensional (2D) convolutional neural networks (CNNs) and three-dimensional (3D) CNNs have been successfully used for analyzing volumetric data in many state-of-the-art medical imaging applications. We propose an alternative modular framework that analyzes volumetric data with an approach that is analogous to radiologists' interpretation, and apply the framework to reduce false positives that are generated in computer-aided detection (CADe) systems for pulmonary nodules in thoracic computed tomography (CT) scans. Methods: In our approach, a deep network consisting of 2D CNNs first processes slices individually. The features extracted in this stage are then passed to a recurrent neural network (RNN), thereby modeling consecutive slices as a sequence of temporal data and capturing the contextual information across all three dimensions in the volume of interest. Outputs of the RNN layer are weighed before the final fully connected layer, enabling the network to scale the importance of different slices within a volume of interest in an end-to-end training framework. Results: We validated the proposed architecture on the false positive reduction track of the lung nodule analysis (LUNA) challenge for pulmonary nodule detection in chest CT scans, and obtained competitive results compared to 3D CNNs. Our results show that the proposed approach can encode the 3D information in volumetric data effectively by achieving a sensitivity >0.8 with just 1/8 false positives per scan. Conclusions: Our experimental results demonstrate the effectiveness of temporal analysis of volumetric images for the application of false positive reduction in chest CT scans and show that state-ofthe-art 2D architectures from the literature can be directly applied to analyzing volumetric medical data. As newer and better 2D architectures are being developed at a much faster rate compared to 3D architectures, our approach makes it easy to obtain state-of-the-art performance on volumetric data using new 2D architectures.

Automatic lung nodule detection in thoracic CT scans using dilated slice‐wise convolutions

“…The comparison is reported in terms of sensitivities at seven different operating points and summarized at the last row using the CPM metric. The CAD system developed by Setio et al 5 used multiple-view 2D CNNs, our method and Farhangi et al 27 are based on methods of temporal analysis of CT slices, and 3D CNNs were used in rest of studies 11,13,18,19,21 in this table. It is observed that our method achieves the highest sensitivity at an average of 8 FPs/scan, but it is not superior to be best algorithms at the lowest FP rates.…”

“…The CAD system developed by Setio et al 5 . used multiple‐view 2D CNNs, our method and Farhangi et al 27 . are based on methods of temporal analysis of CT slices, and 3D CNNs were used in rest of studies 11,13,18,19,21 in this table.…”

“…Screening and FP Reduction Dou et al 18 Farhangi et al 27 Setio et al 5 Dou et al 13 Pezeshk et al 19 Cao et al 21 Zheng et al 11 available training samples and the heavily imbalanced nature of data in medical imaging. In this study, we propose an alternative approach to multiview 2D or 3D CNNs by a combination of 2D and 1D convolutions in a slice-wise manner to produce an efficient 3D implementation.…”

“…This approach reduces the issues associated with a smaller number of training samples relative to the number of parameters in the network by using existing 2D architectures while achieving competitive results. 27 Here, we extend the idea of temporal analysis of VOIs in the axial direction for both the screening and false positive reduction stages of our CADe system. However, the direct application of RNN networks to the entire CT volume is not straightforward; instead, based on recent results from Bai et al 29 that showed 1D CNNs can outperform recurrent networks on tasks such as audio synthesis and machine translation, we similarly demonstrate that interslice dependencies can be effectively captured using dilated 1D convolutions that operate across the slices in the CT volumes.…”

“…To this end, recurrent networks are the standard choice due to their robust performance in a variety of applications, such as speech 25 and activity recognition 26 . In our previous study, 27 we explored the potential of recurrent neural networks (RNNs) in the false positive reduction stage, and processed the slices in each candidate VOI in a temporal manner using gated recurrent units (GRUs) 28 . The intuition behind modeling slice sequences using GRUs is that the hidden states in the memory units of GRUs can retain the information in the sequence of slices within a volume.…”