Transfer learning radiomics based on multimodal ultrasound imaging for staging liver fibrosis

Abstract

Objectives To propose a transfer learning (TL) radiomics model that efficiently combines the information from gray scale and elastogram ultrasound images for accurate liver fibrosis grading. Methods Totally 466 patients undergoing partial hepatectomy were enrolled, including 401 with chronic hepatitis B and 65 without fibrosis pathologically. All patients received elastography and got liver stiffness measurement (LSM) 2-3 days before surgery. We proposed a deep convolutional neural network by TL to analyze images of gray scale modality (GM) and elastogram modality (EM). The TL process was used for liver fibrosis classification by Inception-V3 network which pretrained on ImageNet. The diagnostic performance of TL and non-TL was compared. The value of single modalities, including GM and EM alone, and multimodalities, including GM + LSM and GM + EM, was evaluated and compared with that of LSM and serological indexes. Receiver operating characteristic curve analysis was performed to calculate the optimal area under the curve (AUC) for classifying fibrosis of S4, ≥ S3, and ≥ S2. Results TL in GM and EM demonstrated higher diagnostic accuracy than non-TL, with significantly higher AUCs (all p < .01). Single-modal GM and EM both performed better than LSM and serum indexes (all p < .001). Multimodal GM + EM was the most accurate prediction model (AUCs are 0.950, 0.932, and 0.930 for classifying S4, ≥ S3, and ≥ S2, respectively) compared with GM + LSM, GM and EM alone, LSM, and biomarkers (all p < .05). Conclusions Liver fibrosis can be staged by a transfer learning modal based on the combination of gray scale and elastogram ultrasound images, with excellent performance. Key Points • Transfer learning consists in applying to a specific deep learning algorithm that pretrained on another relevant problem, expected to reduce the risk of overfitting due to insufficient medical images. • Liver fibrosis can be staged by transfer learning radiomics with excellent performance. • The most accurate prediction model of transfer learning by Inception-V3 network is the combination of gray scale and elastogram ultrasound images.

Artificial intelligence in precision medicine in hepatology

“…Radiomics has been used to evaluate chronic liver disease and assess the prognosis of malignant liver tumors. Like segmentation, the accuracy for diagnosing parenchymal liver disease‐related parameters 32,33 is higher than malignant tumors‐related parameters 34,35 . For example, ultrasonography and MRI‐based radiomics features have been accurate for staging liver fibrosis.…”

Technology trends and applications of deep learning in ultrasonography: image quality enhancement, diagnostic support, and improving workflow efficiency

“…Many networks have been proposed for automatic liver fibrosis staging, with examples including a four-layer CNN with elastographic image input [ 68 ] and a METAVIR [ 69 , 70 ] score prediction network from B-mode images. Xue et al [ 71 ] used a multiple modality input of B-mode and elastography images. Two networks were trained using B-mode and elastography individually, and the results of each were combined to generate fibrosis staging.…”

Current status of deep learning applications in abdominal ultrasonography

“…Ten studies applying deep learning to liver US imaging aimed to evaluate diffuse liver disease, especially hepatic fibrosis and steatosis [2][3][4][5][6][7][8][9][10][11]. These studies are summarized in Table 1.…”