Reinventing 2D Convolutions for 3D Images

Yang, Jiancheng; Huang, Xiaoyang; Yi, He; Xu, Jingwei; Yang, Canqian; Ni, Bingbing

doi:10.1109/jbhi.2021.3049452

Cited by 73 publications

(53 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It can only be adopted to capture spatial information from multiple axial slices. To make up for the inconsistent representation ability along the depth axis and the height/width axis, ACS [20] further performed a symmetric pseudo-3D convolution operation. It expanded the dimensions of 2D convolution kernel along depth, height and width axis respectively to generate three view-based 3D convolution kernels.…”

Section: A Transfer Learning From 2d Weightsmentioning

confidence: 99%

“…For fully-supervised methods, Med3D 3 [30] and 3D ResNet pretrained on Kinetics 4 [22] are included for comparison, both of which show impressive performance and have released their pre-trained weights to the public. For methods that convert 2D pre-trained weights, we compare our models with the classic I3D [22] and the newly proposed ACS 5 [20], which have achieved competitive performance on many 3D medical imaging tasks [16]. For all the compared methods, we use their publicly released model architecture and pre-trained weights in all our experiments.…”

Section: B Transfer To 3d Medical Imaging Tasksmentioning

confidence: 99%

“…Following the prior works [20], [21], only nodules with diameters ≥ 3mm are adopted in our study, since smaller nodules are not considered to be clinically relevant by current screening protocols. For both tasks, we apply the same data split as used in ACS [20] for model training and evaluation. Specifically, for lung nodule segmentation, a total of 2,142 samples are used for training and 526 samples for testing.…”

Section: Lung Nodule Classification and Segmentationmentioning

confidence: 99%

“…2) Performance Evaluation: The experimental results of different methods on lung nodule classification task are shown in Table IV. As done in [24] and [20], we also employ AUC as the main compared evaluation metric. It reflects the overall performance of a binary classifier and is not affected by class imbalance problems.…”

Section: Lung Nodule Classification and Segmentationmentioning

confidence: 99%

See 3 more Smart Citations

Advancing 3D Medical Image Analysis with Variable Dimension Transform based Supervised 3D Pre-training

Zhang¹,

Li²,

Zhou³

et al. 2022

Preprint

View full text Add to dashboard Cite

The difficulties in both data acquisition and annotation substantially restrict the sample sizes of training datasets for 3D medical imaging applications. As a result, constructing high-performance 3D convolutional neural networks from scratch remains a difficult task in the absence of a sufficient pre-training parameter. Previous efforts on 3D pre-training have frequently relied on selfsupervised approaches, which use either predictive or contrastive learning on unlabeled data to build invariant 3D representations. However, because of the unavailability of large-scale supervision information, obtaining semantically invariant and discriminative representations from these learning frameworks remains problematic. In this paper, we revisit an innovative yet simple fully-supervised 3D network pre-training framework to take advantage of semantic supervisions from large-scale 2D natural image datasets. With a redesigned 3D network architecture, reformulated natural images are used to address the problem of data scarcity and develop powerful 3D representations. Comprehensive experiments on four benchmark datasets demonstrate that the proposed pre-trained models can effectively accelerate convergence while also improving accuracy for a variety of 3D medical imaging tasks such as classification, segmentation and detection. In addition, as compared to training from scratch, it can save up to 60% of annotation efforts. On the NIH DeepLesion dataset, it likewise achieves state-ofthe-art detection performance, outperforming earlier selfsupervised and fully-supervised pre-training approaches, as well as methods that do training from scratch. To facilitate further development of 3D medical models, our code and pre-trained model weights are publicly available at https://github.com/urmagicsmine/CSPR.

show abstract

Section: A Transfer Learning From 2d Weightsmentioning

confidence: 99%

Section: B Transfer To 3d Medical Imaging Tasksmentioning

confidence: 99%

Section: Lung Nodule Classification and Segmentationmentioning

confidence: 99%

Section: Lung Nodule Classification and Segmentationmentioning

confidence: 99%

See 2 more Smart Citations

Advancing 3D Medical Image Analysis with Variable Dimension Transform based Supervised 3D Pre-training

Zhang¹,

Li²,

Zhou³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Recently, there have been a family of techniques that enable building 3D networks with 2D pretraining [2,13,23,9,22], we refer to it as 3D context fusion operators. See Sec.…”

Section: Introductionmentioning

confidence: 99%

Asymmetric 3D Context Fusion for Universal Lesion Detection

Yang,

He,

Kuang

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Modeling 3D context is essential for high-performance 3D medical image analysis. Although 2D networks benefit from large-scale 2D supervised pretraining, it is weak in capturing 3D context. 3D networks are strong in 3D context yet lack supervised pretraining. As an emerging technique, 3D context fusion operator, which enables conversion from 2D pretrained networks, leverages the advantages of both and has achieved great success. Existing 3D context fusion operators are designed to be spatially symmetric, i.e., performing identical operations on each 2D slice like convolutions. However, these operators are not truly equivariant to translation, especially when only a few 3D slices are used as inputs. In this paper, we propose a novel asymmetric 3D context fusion operator (A3D), which uses different weights to fuse 3D context from different 2D slices. Notably, A3D is NOT translation-equivariant while it significantly outperforms existing symmetric context fusion operators without introducing large computational overhead. We validate the effectiveness of the proposed method by extensive experiments on DeepLesion benchmark, a large-scale public dataset for universal lesion detection from computed tomography (CT). The proposed A3D consistently outperforms symmetric context fusion operators by considerable margins, and establishes a new state of the art on DeepLesion. To facilitate open research, our code and model in PyTorch is available at https://github.com/M3DV/AlignShift.

show abstract

AAPM task group report 273: Recommendations on best practices for AI and machine learning for computer‐aided diagnosis in medical imaging

et al. 2023

View full text Add to dashboard Cite

Rapid advances in artificial intelligence (AI) and machine learning, and specifically in deep learning (DL) techniques, have enabled broad application of these methods in health care. The promise of the DL approach has spurred further interest in computer‐aided diagnosis (CAD) development and applications using both “traditional” machine learning methods and newer DL‐based methods. We use the term CAD‐AI to refer to this expanded clinical decision support environment that uses traditional and DL‐based AI methods. Numerous studies have been published to date on the development of machine learning tools for computer‐aided, or AI‐assisted, clinical tasks. However, most of these machine learning models are not ready for clinical deployment. It is of paramount importance to ensure that a clinical decision support tool undergoes proper training and rigorous validation of its generalizability and robustness before adoption for patient care in the clinic. To address these important issues, the American Association of Physicists in Medicine (AAPM) Computer‐Aided Image Analysis Subcommittee (CADSC) is charged, in part, to develop recommendations on practices and standards for the development and performance assessment of computer‐aided decision support systems. The committee has previously published two opinion papers on the evaluation of CAD systems and issues associated with user training and quality assurance of these systems in the clinic. With machine learning techniques continuing to evolve and CAD applications expanding to new stages of the patient care process, the current task group report considers the broader issues common to the development of most, if not all, CAD‐AI applications and their translation from the bench to the clinic. The goal is to bring attention to the proper training and validation of machine learning algorithms that may improve their generalizability and reliability and accelerate the adoption of CAD‐AI systems for clinical decision support.

show abstract

Reinventing 2D Convolutions for 3D Images

Cited by 73 publications

References 50 publications

Advancing 3D Medical Image Analysis with Variable Dimension Transform based Supervised 3D Pre-training

Advancing 3D Medical Image Analysis with Variable Dimension Transform based Supervised 3D Pre-training

Asymmetric 3D Context Fusion for Universal Lesion Detection

AAPM task group report 273: Recommendations on best practices for AI and machine learning for computer‐aided diagnosis in medical imaging

Contact Info

Product

Resources

About