MVP-Net: Multi-view FPN with Position-Aware Attention for Deep Universal Lesion Detection

Abstract

Universal lesion detection (ULD) on computed tomography (CT) images is an important but underdeveloped problem. Recently, deep learning-based approaches have been proposed for ULD, aiming to learn representative features from annotated CT data. However, the hunger for data of deep learning models and the scarcity of medical annotation hinders these approaches to advance further. In this paper, we propose to incorporate domain knowledge in clinical practice into the model design of universal lesion detectors. Specifically, as radiologists tend to inspect multiple windows for an accurate diagnosis, we explicitly model this process and propose a multi-view feature pyramid network (FPN), where multi-view features are extracted from images rendered with varied window widths and window levels; to effectively combine this multi-view information, we further propose a position-aware attention module. With the proposed model design, the data-hunger problem is relieved as the learning task is made easier with the correctly induced clinical practice prior. We show promising results with the proposed model, achieving an absolute gain of 5.65% (in the sensitivity of FPs@4.0) over the previous state-of-the-art on the NIH DeepLesion dataset.

A deep learning approach to characterize 2019 coronavirus disease (COVID-19) pneumonia in chest CT images

“…Noteworthy, radiologists across the world have provided new insights by accessing the lung CT as additional diagnosis or screening tool of COVID-19 pneumonia. Basically, bilateral GGOs, consolidative pulmonary opacities, as well as the prominent subpleural distribution are regarded as classical features in chest CT images of patients diagnosed with COVID-19 pneumonia, which are similar to those reported with SARS-CoV and MERS-CoV [9][10][11][12][13][14][15][16][17][18][19]. In parallel with these findings, our study also demonstrated higher incidence of GGOs and consolidative opacities in the CT images from COVID pneumonia patients.…”

“…In this study, COVID-19 pneumonia-based lung lesions included consolidation, GGO, nodules and others such as fibrosis. A convolutional MVP-Net [18] is exploited to achieve automatic detection of the lesions. Domain knowledge is incorporated in clinical practice during the model design.…”

“…To effectively combine this multi-view information, a channel-wise attention module is employed to capture complementary information across different views. The overall architecture of the network is shown in our previous published work [18]. A threepathway architecture is built to extract the most prominent features from each representative view, followed by a classifier and regressor to classify and localize the potential abnormal regions in CT images.…”

Dynamic evaluation of lung involvement during coronavirus disease-2019 (COVID-19) with quantitative lung CT

“…Four patients (4/18, 22%) were classified into mild type without any abnormalities on chest CT scans. Ten patients (10/18, 55.6%) were in moderate type and four patients (4/18, 22%) in severe type according to the preliminary diagnosis and treatment protocol issued by the National Health Commission of the People's Republic of China [13]. There were no cases of critical type in **** City (a city located in the southwest of China with 3.9 million population).…”

“…Since radiologists tend to inspect multiple windows to obtain an accurate diagnosis, the MVP-Net takes advantage of such domain knowledge and employs a multi-view feature pyramid network to extract features from images rendered with varying window widths and window levels. Afterward, 3D U-Net [13] with pseudo-3D convolution was introduced to segment voxels that represented the abnormality in the detected regions. Thus, we could acquire the delineation of the pneumonia-related symptom regions.…”