2.5D CNN model for detecting lung disease using weak supervision

Geng, Yingsan; Ren, Yinhao; Hou, Rui; Han, Songyue; Rubin, Geoffrey D.; Lo, Joseph Y.

doi:10.1117/12.2513631

Cited by 6 publications

(8 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One of the more common methods, usually called 2.5D, is to use CNNs that combine triplanar 2D CNNs from intersecting orthogonal patches. [23][24][25][26][27][28][29][30] This can be a computationally efficient way to incorporate more 3D spatial information, and these studies all present promising results. However, this method is limited in the volumetric information it can encompass at once, since it employs only three orthogonal planes to provide spatial information for a single voxel.…”

Section: A Related Workmentioning

confidence: 99%

“…While these studies have shown that 3D CNNs are worth the effort, alternative approaches have been investigated to involve volumetric context to improve segmentation while avoiding 3D convolutions altogether. One of the more common methods, usually called 2.5D, is to use CNNs that combine triplanar 2D CNNs from intersecting orthogonal patches 23–30 . This can be a computationally efficient way to incorporate more 3D spatial information, and these studies all present promising results.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of multislice inputs to convolutional neural networks for medical image segmentation

Grimbergen

Nyholm

et al. 2020

Medical Physics

View full text Add to dashboard Cite

Purpose When using convolutional neural networks (CNNs) for segmentation of organs and lesions in medical images, the conventional approach is to work with inputs and outputs either as single slice [two‐dimensional (2D)] or whole volumes [three‐dimensional (3D)]. One common alternative, in this study denoted as pseudo‐3D, is to use a stack of adjacent slices as input and produce a prediction for at least the central slice. This approach gives the network the possibility to capture 3D spatial information, with only a minor additional computational cost. Methods In this study, we systematically evaluate the segmentation performance and computational costs of this pseudo‐3D approach as a function of the number of input slices, and compare the results to conventional end‐to‐end 2D and 3D CNNs, and to triplanar orthogonal 2D CNNs. The standard pseudo‐3D method regards the neighboring slices as multiple input image channels. We additionally design and evaluate a novel, simple approach where the input stack is a volumetric input that is repeatably convolved in 3D to obtain a 2D feature map. This 2D map is in turn fed into a standard 2D network. We conducted experiments using two different CNN backbone architectures and on eight diverse data sets covering different anatomical regions, imaging modalities, and segmentation tasks. Results We found that while both pseudo‐3D methods can process a large number of slices at once and still be computationally much more efficient than fully 3D CNNs, a significant improvement over a regular 2D CNN was only observed with two of the eight data sets. triplanar networks had the poorest performance of all the evaluated models. An analysis of the structural properties of the segmentation masks revealed no relations to the segmentation performance with respect to the number of input slices. A post hoc rank sum test which combined all metrics and data sets yielded that only our newly proposed pseudo‐3D method with an input size of 13 slices outperformed almost all methods. Conclusion In the general case, multislice inputs appear not to improve segmentation results over using 2D or 3D CNNs. For the particular case of 13 input slices, the proposed novel pseudo‐3D method does appear to have a slight advantage across all data sets compared to all other methods evaluated in this work.

show abstract

Section: A Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Evaluation of multislice inputs to convolutional neural networks for medical image segmentation

Grimbergen

Nyholm

et al. 2020

Medical Physics

View full text Add to dashboard Cite

show abstract

“…While these studies have shown that 3D CNNs are worth the effort, alternative approaches have been investigated to involve volumetric context to improve segmentation while avoiding 3D convolutions altogether. One of the more common methods, usually called 2.5D, is to use CNNs that combine tri-planar 2D CNNs from intersecting orthogonal patches [23,24,25,26,27,28,29,30]. This can be a computationally efficient way of incorporating more 3D spatial information, and these studies all present promising results.…”

Section: A Related Workmentioning

confidence: 99%

Evaluation of Multi-Slice Inputs to Convolutional Neural Networks for Medical Image Segmentation

Vu,

Grimbergen,

Nyholm

et al. 2019

Preprint

View full text Add to dashboard Cite

When using Convolutional Neural Networks (CNNs) for segmentation of organs and lesions in medical images, the conventional approach is to work with inputs and outputs either as single slice (2D) or whole volumes (3D). One common alternative, in this study denoted as pseudo-3D, is to use a stack of adjacent slices as input and produce a prediction for at least the central slice. This approach gives the network the possibility to capture 3D spatial information, with only a minor additional computational cost.In this study, we systematically evaluate the segmentation performance and computational costs of this pseudo-3D approach as a function of the number of input slices, and compare the results to conventional end-to-end 2D and 3D CNNs. The standard pseudo-3D method regards the neighboring slices as multiple input image channels. We additionally evaluate a simple approach where the input stack is a volumetric input that is repeatably convolved in 3D to obtain a 2D feature map. This 2D map is in turn fed into a standard 2D network. We conducted experiments using two different CNN backbone architectures and on five diverse data sets covering different anatomical regions, imaging modalities, and segmentation tasks.We found that while both pseudo-3D methods can process a large number of slices at once and still be computationally much more efficient than fully 3D CNNs, a significant improvement over a regular 2D CNN was only observed for one of the five data sets. An analysis of the structural properties of the segmentation masks revealed no relations to the segmentation performance with respect to the number of input slices.The conclusion is therefore that in the general case, multi-slice inputs appear to not significantly improve segmentation results over using 2D or 3D CNNs.

show abstract

“…Training set image annotation was achieved by a combination of traditional human expert labeling (24 178 images labeled by 28 radiologists and 4th-year residents) and natural language processing (564 975 images labeled by using information extracted via a trained deep learning natural language processing algorithm from corresponding radiology reports). Application of both human expert data labeling (an example of strong supervision) and natural language processing-enabled extraction of image labels from radiology reports (an example of weak supervision) is an increasingly popular way to generate large numbers of data for training (4,5).…”

mentioning

confidence: 99%

“…There needs to be a better more scalable way to "drill for gas." Early but promising work (including this study) in using weak supervision by using labels extracted from radiology, pathologic reports, and other unstructured narrative reports leveraging machine learning natural language processing should help (4,5). This approach exploits the fact that our radiology reports are essentially annotations of the clinical image dataset, albeit annotations with great variability.…”

mentioning

confidence: 99%

Moving Artificial Intelligence from Feasible to Real: Time to Drill for Gas and Build Roads

Chang

2020

Radiology

View full text Add to dashboard Cite

2.5D CNN model for detecting lung disease using weak supervision

Cited by 6 publications

References 1 publication

Evaluation of multislice inputs to convolutional neural networks for medical image segmentation

Evaluation of multislice inputs to convolutional neural networks for medical image segmentation

Evaluation of Multi-Slice Inputs to Convolutional Neural Networks for Medical Image Segmentation

Moving Artificial Intelligence from Feasible to Real: Time to Drill for Gas and Build Roads

Contact Info

Product

Resources

About