Artificial Intelligent Multi-Modal Point-of-Care System for Predicting Response of Transarterial Chemoembolization in Hepatocellular Carcinoma

Sun, Zhonghua; Shi, Zhongxing; Xin, Yanjie; Zhao, Sheng; Jiang, Hao; Wang, Dandan; Zhang, Linhan; Wang, Ziao; Dai, Yanmei; Jiang, Huijie

doi:10.3389/fbioe.2021.761548

Cited by 2 publications

(13 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The studies included in this analysis employ multimodal deep-learning methods for the diagnosis and prediction of liver cancer. State-of-the-art deep learning models such as VGG19 and DeepAttnMISL, which is a state-of-the-art are employed for recognizing image modalities (Hou et al, 2022 ), while GhostNet/CNN, a combination of two single deep learning neural network models, is used for predicting treatment response to trans-arterial chemoembolization (TACE) (Sun et al, 2021 ).…”

Section: Resultsmentioning

confidence: 99%

“…The 10 studies discussed four different types of medical images, explained in Figure 2 . The most common imaging modality used in the studies was CT used in six studies (Liu et al, 2020 ; Menegotto et al, 2020 , 2021 ; Fu et al, 2021 ; Gao et al, 2021 ; Sun et al, 2021 ), followed by MRI used in two studies (Zhen et al, 2020 ; Song et al, 2021 ), WSI used in one study (Hou et al, 2022 ), and Digital Subtraction Angiography (DSA) used in one study (Zhang et al, 2022 ), respectively. All these studies used single image modality and did not report combined use of multiple imaging modalities.…”

Section: Resultsmentioning

confidence: 99%

“…Prediction was the most common purpose for the use of multimodal AI techniques, addressed in six studies (Liu et al, 2020 ; Fu et al, 2021 ; Song et al, 2021 ; Sun et al, 2021 ; Hou et al, 2022 ; Zhang et al, 2022 ), while four studies (Menegotto et al, 2020 , 2021 ; Zhen et al, 2020 ; Gao et al, 2021 ) used multimodal AI techniques for the purposes of diagnosis or classification of HCC (Menegotto et al, 2020 , 2021 ; Zhen et al, 2020 ; Gao et al, 2021 ). The HCC predicting multimodal AI techniques are sub-categorized based on the type to survival prediction (Hou et al, 2022 ), Tran's catheter arterial chemoembolization (TACE) treatment response prediction (Sun et al, 2021 ), and microvascular invasion (MVI) prediction (Fu et al, 2021 ; Song et al, 2021 ). One study (Liu et al, 2020 ) introduced a multimodal AI model capable of performing all three types of predictions.…”

Section: Resultsmentioning

confidence: 99%

“…One study (Liu et al, 2020 ) introduced a multimodal AI model capable of performing all three types of predictions. For prediction purpose, the commonly used imaging modality was MRI and CT scans used in two studies (Song et al, 2021 ; Sun et al, 2021 ), followed by WSI and DSA, each used in one study (Hou et al, 2022 ; Zhang et al, 2022 ), respectively.…”

Section: Resultsmentioning

confidence: 99%

“…Xception CNN used 20 clinical parameters (Menegotto et al, 2021 ), Google Inception-ResNet-V2 CNN used 16 clinical parameters (Zhen et al, 2020 ). GhostNet/CNN used 14 clinical parameters (Sun et al, 2021 ). Cox-Proportional Hazard (Cox-PH), CNN + Gated recurrent neural network (RNN) Spatial Extractor-Temporal Encoder-Integration-Classifier (STIC), and Cox proportional hazards model had the lowest number of EHR modalities, i.e., 9, 8, and 3, reported in Liu et al ( 2020 ), Gao et al ( 2021 ), and Hou et al ( 2022 ), respectively.…”

Section: Resultsmentioning

confidence: 99%

See 4 more Smart Citations

Multimodal deep learning for liver cancer applications: a scoping review

Siam,

Alsaify,

Mohammad

et al. 2023

Front. Artif. Intell.

View full text Add to dashboard Cite

BackgroundHepatocellular carcinoma is a malignant neoplasm of the liver and a leading cause of cancer-related deaths worldwide. The multimodal data combines several modalities, such as medical images, clinical parameters, and electronic health record (EHR) reports, from diverse sources to accomplish the diagnosis of liver cancer. The introduction of deep learning models with multimodal data can enhance the diagnosis and improve physicians' decision-making for cancer patients.ObjectiveThis scoping review explores the use of multimodal deep learning techniques (i.e., combining medical images and EHR data) in diagnosing and prognosis of hepatocellular carcinoma (HCC) and cholangiocarcinoma (CCA).MethodologyA comprehensive literature search was conducted in six databases along with forward and backward references list checking of the included studies. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) extension for scoping review guidelines were followed for the study selection process. The data was extracted and synthesized from the included studies through thematic analysis.ResultsTen studies were included in this review. These studies utilized multimodal deep learning to predict and diagnose hepatocellular carcinoma (HCC), but no studies examined cholangiocarcinoma (CCA). Four imaging modalities (CT, MRI, WSI, and DSA) and 51 unique EHR records (clinical parameters and biomarkers) were used in these studies. The most frequently used medical imaging modalities were CT scans followed by MRI, whereas the most common EHR parameters used were age, gender, alpha-fetoprotein AFP, albumin, coagulation factors, and bilirubin. Ten unique deep-learning techniques were applied to both EHR modalities and imaging modalities for two main purposes, prediction and diagnosis.ConclusionThe use of multimodal data and deep learning techniques can help in the diagnosis and prediction of HCC. However, there is a limited number of works and available datasets for liver cancer, thus limiting the overall advancements of AI for liver cancer applications. Hence, more research should be undertaken to explore further the potential of multimodal deep learning in liver cancer applications.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

See 3 more Smart Citations

Multimodal deep learning for liver cancer applications: a scoping review

Siam,

Alsaify,

Mohammad

et al. 2023

Front. Artif. Intell.

View full text Add to dashboard Cite

show abstract

Age-Related Macular Degeneration Detection in Retinal Fundus Images by a Deep Convolutional Neural Network

García-Floriano,

Ventura-Molina

2024

Mathematics

View full text Add to dashboard Cite

Computer-based pre-diagnosis of diseases through medical imaging is a task worked on for many years. The so-called fundus images stand out since they do not have uniform illumination and are highly sensitive to noise. One of the diseases that can be pre-diagnosed through fundus images is age-related macular degeneration, which initially manifests as the appearance of lesions called drusen. Several ways of pre-diagnosing macular degeneration have been proposed, methods based entirely on the segmentation of drusen with prior image processing have been designed and applied, and methods based on image pre-processing and subsequent conversion to feature vectors, or patterns, to be classified by a Machine-Learning model have also been developed. Finally, in recent years, the use of Deep-Learning models, particularly Convolutional Networks, has been proposed and used in classification problems where the data are only images. The latter has allowed the so-called transfer learning, which consists of using the learning achieved in the solution of one problem to solve another. In this paper, we propose the use of transfer learning through the Xception Deep Convolutional Neural Network to detect age-related macular degeneration in fundus images. The performance of the Xception model was compared against six other state-of-the-art models with a dataset created from images available in public and private datasets, which were divided into training/validation and test; with the training/validation set, the training was made using 10-fold cross-validation. The results show that the Xception neural network obtained a validation accuracy that surpasses other models, such as the VGG-16 or VGG-19 networks, and had an accuracy higher than 80% in the test set. We consider that the contributions of this work include the use of a Convolutional Neural Network model for the detection of age-related macular degeneration through the classification of fundus images in those affected by AMD (drusen) and the images of healthy patients. The performance of this model is compared against other methods featured in the state-of-the-art approaches, and the best model is tested on a test set outside the training and validation set.

show abstract

Artificial Intelligent Multi-Modal Point-of-Care System for Predicting Response of Transarterial Chemoembolization in Hepatocellular Carcinoma

Cited by 2 publications

References 35 publications

Multimodal deep learning for liver cancer applications: a scoping review

Multimodal deep learning for liver cancer applications: a scoping review

Age-Related Macular Degeneration Detection in Retinal Fundus Images by a Deep Convolutional Neural Network

Contact Info

Product

Resources

About