Deep Learning-Based Computer-Aided Fetal Echocardiography: Application to Heart Standard View Segmentation for Congenital Heart Defects Detection

Abstract

Accurate segmentation of fetal heart in echocardiography images is essential for detecting the structural abnormalities such as congenital heart defects (CHDs). Due to the wide variations attributed to different factors, such as maternal obesity, abdominal scars, amniotic fluid volume, and great vessel connections, this process is still a challenging problem. CHDs detection with expertise in general are substandard; the accuracy of measurements remains highly dependent on humans’ training, skills, and experience. To make such a process automatic, this study proposes deep learning-based computer-aided fetal heart echocardiography examinations with an instance segmentation approach, which inherently segments the four standard heart views and detects the defect simultaneously. We conducted several experiments with 1149 fetal heart images for predicting 24 objects, including four shapes of fetal heart standard views, 17 objects of heart-chambers in each view, and three cases of congenital heart defect. The result showed that the proposed model performed satisfactory performance for standard views segmentation, with a 79.97% intersection over union and 89.70% Dice coefficient similarity. It also performed well in the CHDs detection, with mean average precision around 98.30% for intra-patient variation and 82.42% for inter-patient variation. We believe that automatic segmentation and detection techniques could make an important contribution toward improving congenital heart disease diagnosis rates.

Automated Precancerous Lesion Screening Using an Instance Segmentation Technique for Improving Accuracy

“…The networks’ original architecture of mask-RCNN was maintained in all cervicograms. All networks were first pretrained using the Microsoft Common Objects in Context (COCO) dataset [ 25 , 26 ], and then fully retrained using our training data to produce the probability scores for each class.…”

“…In the segmentation process, the anatomy landmark of the cervix plays an important role, especially in SCJ, CA, TZ, and AW lesions. Two gynecological oncologists who have over 10 years of experience following the above protocol manually annotated such landmarks using an annotation tool (LabelMe) as the ground truth [ 25 , 26 ]. Both normal (without AW lesion) and abnormal (with AW lesion) cervicogram samples had significant variations in image quality, shape, size, and orientation ( Figure 3 ).…”

“…The instance segmentation approach was developed in our previous study and it is based on the mask-RCNN architecture here [ 25 , 26 ]. This architecture has two main parts: the region proposal networks (RPNs) that are used as feature extraction processes from raw data, and fully convolutional networks (FCNs) that function as a multitask learning process for simultaneous classification, detection, and segmentation ( Figure 5 ).…”

“…This architecture has two main parts: the region proposal networks (RPNs) that are used as feature extraction processes from raw data, and fully convolutional networks (FCNs) that function as a multitask learning process for simultaneous classification, detection, and segmentation ( Figure 5 ). The mask-RCNN structure allows for the system to generate the class, location (boundary box), and shape of the object of interest [ 25 , 26 ]. The proposed model utilizes a multitask loss function that combines the loss of classification, localization, and mask segmentation, as illustrated in Equation (1).…”

“…DSC is a statistical tool that measures the similarity and diversity of sample sets [ 25 , 26 , 27 ]. In this case, we measured the performance of predictive images with detailed truth labels.…”