Joint segmentation and classification of hepatic lesions in ultrasound images using deep learning

Abstract

Objectives To develop a convolutional neural network system to jointly segment and classify a hepatic lesion selected by user clicks in ultrasound images. Methods In total, 4309 anonymized ultrasound images of 3873 patients with hepatic cyst (n = 1214), hemangioma (n = 1220), metastasis (n = 1001), or hepatocellular carcinoma (HCC) (n = 874) were collected and annotated. The images were divided into 3909 training and 400 test images. Our network is composed of one shared encoder and two inference branches used for segmentation and classification and takes the concatenation of an input image and two Euclidean distance maps of foreground and background clicks provided by a user as input. The performance of hepatic lesion segmentation was evaluated based on the Jaccard index (JI), and the performance of classification was based on accuracy, sensitivity, specificity, and the area under the receiver operating characteristic curve (AUROC). Results We achieved performance improvements by jointly conducting segmentation and classification. In the segmentation only system, the mean JI was 68.5%. In the classification only system, the accuracy of classifying four types of hepatic lesions was 79.8%. The mean JI and classification accuracy were 68.5% and 82.2%, respectively, for the proposed joint system. The optimal sensitivity and specificity and the AUROC of classifying benign and malignant hepatic lesions of the joint system were 95.0%, 86.0%, and 0.970, respectively. The respective sensitivity, specificity, and the AUROC for classifying four hepatic lesions of the joint system were 86.7%, 89.7%, and 0.947. Conclusions The proposed joint system exhibited fair performance compared to segmentation only and classification only systems. Key Points • The joint segmentation and classification system using deep learning accurately segmented and classified hepatic lesions selected by user clicks in US examination. • The joint segmentation and classification system for hepatic lesions in US images exhibited higher performance than segmentation only and classification only systems. • The joint segmentation and classification system could assist radiologists with minimal experience in US imaging by characterizing hepatic lesions.

Development and Validation of a Deep Learning Algorithm to Differentiate Colon Carcinoma From Acute Diverticulitis in Computed Tomography Images

“…19 The purely morphologic differentiation of AD from CC based on CT imaging remains difficult. In The potential benefit of AI in gastrointestinal imaging has been demonstrated by several studies [21][22][23] ; however, for CC, mostly histopathologic and endoscopic models exist. 24 In particular, to our knowledge, the differentiation of CC and AD in CT scans by methods of AI has not been investigated so far.…”

The feasibility to use artificial intelligence to aid detecting focal liver lesions in real-time ultrasound: a preliminary study based on videos

“…AI systems for differentiating benign from malignant focal liver lesions is an active area of research. Previous studies have developed AI systems for such characterization task in different imaging modalities, including computerized tomography 21 – 25 , magnetic resonance imaging 26 , 27 , contrast-enhanced ultrasound 28 – 30 and B-mode ultrasound 6 – 9 , 31 – 33 . Lastly, this study was a proof of concept for using the AI system to detect FLLs in ultrasound videos.…”

A systematic review on application of deep learning in digestive system image processing

“…Due to the low contrast between lesions and normal liver tissue, the diagnosis of solid lesions is a challenge. Ryu et al used 4309 US images with focal liver disease, including liver cysts, hemangioma, metastasis, and hepatocellular carcinomas, for DL and precise segmentation and classification of focal liver lesions [ 71 ]. Contrast-Enhanced Ultrasound (CEUS) can allow real-time scanning and provide dynamic perfusion information, so it has the potential to surpass CT and MRI in liver and gallbladder diseases [ 72 , 73 ].…”