MULAN: Multitask Universal Lesion Analysis Network for Joint Lesion Detection, Tagging, and Segmentation

Yan, Ke; Tang, Youbao; Peng, Yifan; Sandfort, Veit; Bagheri, Mohammadhadi; Lu, Zhiyong; Summers, Ronald M.

doi:10.1007/978-3-030-32226-7_22

Cited by 88 publications

(137 citation statements)

References 17 publications

(68 reference statements)

Supporting

Mentioning

135

Contrasting

Order By: Relevance

“…Table 1 presents the comparisons with the previous state-of-the-art (SOTA) methods. Our model surpasses all the SOTA methods on sensitivities at different FPs and MAP@0.5, which includes 3DCE [1], MSB [2], RetinaNet [3], MVP-Net [4] and MULAN [5].…”

Section: Methodsmentioning

confidence: 92%

“…Supervised pre-training from natural images has proven to be an effective way for 2D medical image transfer learning [1][2][3][4][5]10]. This indicates that using supervised pre-training models from another domain can actually benefit the medical image analysis application.…”

Section: Supervised 3d Pre-training With Coco Datasetmentioning

confidence: 99%

“…Yan et al [1] adopts a late fusion strategy which stacked 2D features of neighboring slices to build 3D context enhanced features. Although the pseudo-3D contextual information has provided prominent performance gain [1][2][3][4][5], its late fusion strategy leads to notable losses of context information from early stages of the network. A direct way to address these issues is to employ 3D convolutions which introduce inter-slice connections hierarchically to learn 3D representations end to end.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Revisiting 3D Context Modeling with Supervised Pre-training for Universal Lesion Detection in CT Slices

Zhang

Chen³

et al. 2020

Medical Image Computing and Computer Assisted Intervention – MICCAI 2020

View full text Add to dashboard Cite

Universal lesion detection from computed tomography (CT) slices is important for comprehensive disease screening. Since each lesion can locate in multiple adjacent slices, 3D context modeling is of great significance for developing automated lesion detection algorithms. In this work, we propose a Modified Pseudo-3D Feature Pyramid Network (MP3D FPN) that leverages depthwise separable convolutional filters and a group transform module (GTM) to efficiently extract 3D context enhanced 2D features for universal lesion detection in CT slices. To facilitate faster convergence, a novel 3D network pre-training method is derived using solely large-scale 2D object detection dataset in the natural image domain. We demonstrate that with the novel pre-training method, the proposed MP3D FPN achieves state-of-the-art detection performance on the DeepLesion dataset (3.48% absolute improvement in the sensitivity of FPs@0.5), significantly surpassing the baseline method by up to 6.06% (in MAP@0.5) which adopts 2D convolution for 3D context modeling. Moreover, the proposed 3D pre-trained weights can potentially be used to boost the performance of other 3D medical image analysis tasks.

show abstract

Section: Methodsmentioning

confidence: 92%

Section: Supervised 3d Pre-training With Coco Datasetmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Revisiting 3D Context Modeling with Supervised Pre-training for Universal Lesion Detection in CT Slices

Zhang

Chen³

et al. 2020

Medical Image Computing and Computer Assisted Intervention – MICCAI 2020

View full text Add to dashboard Cite

show abstract

“…As shown in Table 1, our method brings promising detection performance improvements for all baselines. The improvements of Faster R-CNN [27], 9-slice 3DCE, and MVP-Net are more pronounced than those of MULAN w/o SRL [8] and AlignShift [9]. This is because MULAN and AlignShift introduce extra weakly segmentation mask generated from radiologist-annotated RECIST labels.…”

Section: Lesion Detection Performancementioning

confidence: 98%

Bounding Maps for Universal Lesion Detection

Han

Zhou

2020

Medical Image Computing and Computer Assisted Intervention – MICCAI 2020

View full text Add to dashboard Cite

Universal Lesion Detection (ULD) in computed tomography plays an essential role in computer-aided diagnosis. Promising ULD results have been reported by coarse-to-fine two-stage detection approaches, but such two-stage ULD methods still suffer from issues like imbalance of positive v.s. negative anchors during object proposal and insufficient supervision problem during localization regression and classification of the region of interest (RoI) proposals. While leveraging pseudo segmentation masks such as bounding map (BM) can reduce the above issues to some degree, it is still an open problem to effectively handle the diverse lesion shapes and sizes in ULD. In this paper we propose a BM-based conditional training for two-stage ULD, which can (i) reduce positive vs. negative anchor imbalance via a BM-based conditioning (BMC) mechanism for anchor sampling instead of traditional IoU-based rule; and (ii) adaptively compute size-adaptive BM (ABM) from lesion boundingbox, which is used for improving lesion localization accuracy via ABMsupervised segmentation. Experiments with four state-of-the-art methods show that the proposed approach can bring an almost free detection accuracy improvement without requiring expensive lesion mask annotations.

show abstract

“…An illustrative example is, when translating an adult X-ray into a pediatric X-ray, there is no guarantee that fine-grained disease content on the original image will be explicitly transferred. The capability of preserving class-specific semantic context across domains is crucial for medical imaging analysis for certain clinically relevant tasks, such as disease or lesion classification, detection and segmentation [14,11,9,17]. However, to our best knowledge, solutions to this problem of adversarial adaptation for medical imaging are limited.…”

Section: Introductionmentioning

confidence: 99%

TUNA-Net: Task-Oriented UNsupervised Adversarial Network for Disease Recognition in Cross-domain Chest X-rays

Tang

Sandfort

et al. 2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

In this work, we exploit the unsupervised domain adaptation problem for radiology image interpretation across domains. Specifically, we study how to adapt the disease recognition model from a labeled source domain to an unlabeled target domain, so as to reduce the effort of labeling each new dataset. To address the shortcoming of crossdomain, unpaired image-to-image translation methods which typically ignore class-specific semantics, we propose a task-driven, discriminatively trained, cycle-consistent generative adversarial network, termed TUNA-Net. It is able to preserve 1) low-level details, 2) high-level semantic information and 3) mid-level feature representation during the image-toimage translation process, to favor the target disease recognition task. The TUNA-Net framework is general and can be readily adapted to other learning tasks. We evaluate the proposed framework on two public chest X-ray datasets for pneumonia recognition. The TUNA-Net model can adapt labeled adult chest X-rays in the source domain such that they appear as if they were drawn from pediatric X-rays in the unlabeled target domain, while preserving the disease semantics. Extensive experiments show the superiority of the proposed method as compared to state-of-the-art unsupervised domain adaptation approaches. Notably, TUNA-Net achieves an AUC of 96.3% for pediatric pneumonia classification, which is very close to that of the supervised approach (98.1%), but without the need for labels on the target domain.

show abstract

MULAN: Multitask Universal Lesion Analysis Network for Joint Lesion Detection, Tagging, and Segmentation

Cited by 88 publications

References 17 publications

Revisiting 3D Context Modeling with Supervised Pre-training for Universal Lesion Detection in CT Slices

Revisiting 3D Context Modeling with Supervised Pre-training for Universal Lesion Detection in CT Slices

Bounding Maps for Universal Lesion Detection

TUNA-Net: Task-Oriented UNsupervised Adversarial Network for Disease Recognition in Cross-domain Chest X-rays

Contact Info

Product

Resources

About