A survey on incorporating domain knowledge into deep learning for medical image analysis

Xie, Xiao-Liang; Niu, Jianwei; Li, Xuefeng; Chen, Zhengsu; Tang, Shaojie; Yu, Shui

doi:10.1016/j.media.2021.101985

Cited by 177 publications

(134 citation statements)

References 89 publications

Supporting

Mentioning

119

Contrasting

Order By: Relevance

“…Secondly, it describes the specific correlations between data, deep learning models used in the interpretation of diagnostic medical images and their applications in medicine. Finally presents problems and future research directions [3].…”

Section: Introductionmentioning

confidence: 99%

“…In this paper, we aim to achieve an updated characterization of the specifics of the constituent elements of the deep learning process, scientific data, methods of incorporation of knowledge, DL models according to the objectives for which they were designed and presentation of medical applications according to these tasks. Secondly, we will describe the specific correlations between the quality, type and volume of data and their importance in achieving the performance of the deep learning models used in the interpretation of medical diagnostic images [3]. We will also make a structural and functional description of DL models and their applications in medicine.…”

Section: Introductionmentioning

confidence: 99%

“…These medical images are filed in connection with imaging reports or medical video image reports and, along with language processing from natural images, they have a great contribution to image analysis [5]. Annotation and labelling of the medical image, representing data from doctors, used through methods of integration into deep learning models, consumes time and requires specialized knowledge [3].…”

Section: Introductionmentioning

confidence: 99%

“…The large volume of training data and properly labeled determines the performance of the deep learning modeling in the interpretation of medical images [3]. Because manual image labelling requires time and specialized training, standardized, organized labelling has been used which has the risk of over-labeling with unnecessary information [2].…”

Section: Introductionmentioning

confidence: 99%

“…There are many large-scale and well-annotated data sets, such as ImageNet 1 (over 14 million images tagged in 20 k categories) and COCO 2 (with over 200 images annotated in 80 categories), medical datasets (open source), such as ChestX-ray14 and Deep-Lesion containing medical images tagged over 100 k, the others, contain only a few thousand or even hundreds of medical images [3] (Figure 1), and medical applications have developed properly in the medical fields.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Unified Analysis Specific to the Medical Field in the Interpretation of Medical Images through the Use of Deep Learning

Ursuleanu¹,

Luca²,

Gheorghe³

et al. 2021

ETSN

View full text Add to dashboard Cite

Deep learning (DL) has seen an exponential development in recent years, with major impact in many medical fields, especially in the field of medical image. The purpose of the work converges in determining the importance of each component, describing the specificity and correlations of these elements involved in achieving the precision of interpretation of medical images using DL. The major contribution of this work is primarily to the updated characterisation of the characteristics of the constituent elements of the deep learning process, scientific data, methods of knowledge incorporation, DL models according to the objectives for which they were designed and the presentation of medical applications in accordance with these tasks. Secondly, it describes the specific correlations between the quality, type and volume of data, the deep learning patterns used in the interpretation of diagnostic medical images and their applications in medicine. Finally presents problems and directions of future research. Data quality and volume, annotations and labels, identification and automatic extraction of specific medical terms can help deep learning models perform image analysis tasks. Moreover, the development of models capable of extracting unattended features and easily incorporated into the architecture of DL networks and the development of techniques to search for a certain network architecture according to the objectives set lead to performance in the interpretation of medical images.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Unified Analysis Specific to the Medical Field in the Interpretation of Medical Images through the Use of Deep Learning

Ursuleanu¹,

Luca²,

Gheorghe³

et al. 2021

ETSN

View full text Add to dashboard Cite

show abstract

Dual attention guided multiscale neural network trained with curriculum learning for noninvasive prediction of Gleason Grade Group from MRI

Shen

Qiao

et al. 2022

Medical Physics

View full text Add to dashboard Cite

Background The Gleason Grade Group (GG) is essential in assessing the malignancy of prostate cancer (PCa) and is typically obtained by invasive biopsy procedures in which sampling errors could lead to inaccurately scored GGs. With the gradually recognized value of bi‐parametric magnetic resonance imaging (bpMRI) in PCa, it is beneficial to noninvasively predict GGs from bpMRI for early diagnosis and treatment planning of PCa. However, it is challenging to establish the connection between bpMRI features and GGs. Purpose In this study, we propose a dual attention‐guided multiscale neural network (DAMS‐Net) to predict the 5‐scored GG from bpMRI and design a training curriculum to further improve the prediction performance. Methods The proposed DAMS‐Net incorporates a feature pyramid network (FPN) to fully extract the multiscale features for lesions of varying sizes and a dual attention module to focus on lesion and surrounding regions while avoiding the influence of irrelevant ones. Furthermore, to enhance the differential ability for lesions with the inter‐grade similarity and intra‐grade variation in bpMRI, the training process employs a specially designed curriculum based on the differences between the radiological evaluations and the ground truth GGs. Results Extensive experiments were conducted on a private dataset of 382 patients and the public PROSTATEx‐2 dataset. For the private dataset, the experimental results showed that the proposed network performed better than the plain baseline model for GG prediction, achieving a mean quadratic weighted Kappa (Kw) of 0.4902 and a mean positive predictive value of 0.9098 for predicting clinically significant cancer (PPVGG>1). With the application of curriculum learning, the mean Kw and PPVGG>1 further increased to 0.5144 and 0.9118, respectively. For the public dataset, the proposed method achieved state‐of‐the‐art results of 0.5413 Kw and 0.9747 PPVGG>1. Conclusion The proposed DAMS‐Net trained with curriculum learning can effectively predict GGs from bpMRI, which may assist clinicians in early diagnosis and treatment planning for PCa patients.

show abstract

Classification of metastatic hepatic carcinoma and hepatocellular carcinoma lesions using contrast‐enhanced CT based on EI‐CNNet

et al. 2023

View full text Add to dashboard Cite

BackgroundFor hepatocellular carcinoma and metastatic hepatic carcinoma, imaging is one of the main diagnostic methods. In clinical practice, diagnosis mainly relied on experienced imaging physicians, which was inefficient and cannot met the demand for rapid and accurate diagnosis. Therefore, how to efficiently and accurately classify the two types of liver cancer based on imaging is an urgent problem to be solved at present.PurposeThe purpose of this study was to use the deep learning classification model to help radiologists classify the single metastatic hepatic carcinoma and hepatocellular carcinoma based on the enhanced features of enhanced CT (Computer Tomography) portal phase images of the liver site.MethodsIn this retrospective study, 52 patients with metastatic hepatic carcinoma and 50 patients with hepatocellular carcinoma were among the patients who underwent preoperative enhanced CT examinations from 2017–2020. A total of 565 CT slices from these patients were used to train and validate the classification network (EI‐CNNet, training/validation: 452/113). First, the EI block was used to extract edge information from CT slices to enrich fine‐grained information and classify them. Then, ROC (Receiver Operating Characteristic) curve was used to evaluate the performance, accuracy, and recall of the EI‐CNNet. Finally, the classification results of EI‐CNNet were compared with popular classification models.ResultsBy utilizing 80% data for model training and 20% data for model validation, the average accuracy of this experiment was 98.2% ± 0.62 (mean ± standard deviation (SD)), the recall rate was 97.23% ± 2.77, the precision rate was 98.02% ± 2.07, the network parameters were 11.83 MB, and the validation time was 9.83 s/sample. The classification accuracy was improved by 20.98% compared to the base CNN network and the validation time was 10.38 s/sample. Compared with other classification networks, the InceptionV3 network showed improved classification results, but the number of parameters was increased and the validation time was 33 s/sample, and the classification accuracy was improved by 6.51% using this method.ConclusionEI‐CNNet demonstrated promised diagnostic performance and has potential to reduce the workload of radiologists and may help distinguish whether the tumor is primary or metastatic in time; otherwise, it may be missed or misjudged.

show abstract

A survey on incorporating domain knowledge into deep learning for medical image analysis

Cited by 177 publications

References 89 publications

Unified Analysis Specific to the Medical Field in the Interpretation of Medical Images through the Use of Deep Learning

Unified Analysis Specific to the Medical Field in the Interpretation of Medical Images through the Use of Deep Learning

Dual attention guided multiscale neural network trained with curriculum learning for noninvasive prediction of Gleason Grade Group from MRI

Classification of metastatic hepatic carcinoma and hepatocellular carcinoma lesions using contrast‐enhanced CT based on EI‐CNNet

Contact Info

Product

Resources

About