Automated cartilage and meniscus segmentation of knee MRI with conditional generative adversarial networks

Abstract

Purpose Fully automatic tissue segmentation is an essential step to translate quantitative MRI techniques to clinical setting. The goal of this study was to develop a novel approach based on the generative adversarial networks for fully automatic segmentation of knee cartilage and meniscus. Theory and Methods Defining proper loss function for semantic segmentation to enforce the learning of multiscale spatial constraints in an end‐to‐end training process is an open problem. In this work, we have used the conditional generative adversarial networks to improve segmentation performance of convolutional neural network, such as UNet alone by overcoming the problems caused by pixel‐wise mapping based objective functions, and to capture cartilage features during the training of the network. Furthermore, the Dice coefficient and cross entropy losses were incorporated to the loss functions to improve the model performance. The model was trained and tested on 176, 3D DESS (double‐echo steady‐state) knee images from the Osteoarthritis Initiative data set. Results The proposed model provided excellent segmentation performance for cartilages with Dice coefficients ranging from 0.84 in patellar cartilage to 0.91 in lateral tibial cartilage, with an average Dice coefficient of 0.88. For meniscus segmentation, the model achieves 0.89 Dice coefficient for lateral meniscus and 0.87 Dice coefficient for medial meniscus. The results are superior to previously published automatic cartilage and meniscus segmentation methods based on deep learning models such as convolutional neural network. Conclusion The proposed UNet‐conditional generative adversarial networks based model demonstrated a fully automated segmentation method with high accuracy for knee cartilage and meniscus.

Automated magnetic resonance image segmentation of the anterior cruciate ligament

“…To circumvent this bottleneck in the qMRI data processing pipeline, significant effort has been placed into developing automated segmentation methods, especially for joint tissues such as cartilage and meniscus. [16][17][18][19][20] Unfortunately, the ACL has proven to be difficult to segment automatically due to several challenges, including poor contrast and indistinct tissue boundaries relative to background tissue. Perhaps, as a result, the literature on automatic ACL segmentation is scarce.…”

“…These methods have also been successful in cartilage and meniscus segmentation. 17,18 The goal of the present work was to develop a reliable automated segmentation method for the ACL, that could replace manual segmentation in qMRI processing pipelines. A two-dimensional (2D) U-Net 27 architecture was chosen for this task, due to its previous success in MR image segmentation, particularly on other knee structures such as cartilage and meniscus.…”

“…A two-dimensional (2D) U-Net 27 architecture was chosen for this task, due to its previous success in MR image segmentation, particularly on other knee structures such as cartilage and meniscus. 17,18 The Constructive Interference in Steady State (CISS) sequence was selected to develop the automatic segmentation method as it offers relatively high spatial resolution and contrast to enable the algorithm to gain proficiency at the ACL segmentation task. The CISS sequence has been used in clinical trials of ACL healing, 28,29 and has been shown to correlate with the biomechanical properties of healing ACLs and ACL grafts.…”

“…Deep learning has been successfully used for brain image segmentation, 23–26 which presents similar contrast and tissue boundary difficulties to the ACL. These methods have also been successful in cartilage and meniscus segmentation 17,18 …”

Improving Quantitative Magnetic Resonance Imaging Using Deep Learning

“…1,5 In particular, there has been much recent interest in applying deep learning to a wide variety of medical imaging subspecialties including cancer, neurologic, lung, abdomen, breast, and cardiac imaging. [6][7][8] The applications in MSK imaging have emerged, and great success has been achieved for tissue segmentation [9][10][11][12][13][14][15] and image reconstruction. [16][17][18] More recently, the use of deep learning for lesion detection, progression, and prediction of MSK disease on X-ray radiography, computed tomography (CT), and magnetic resonance (MR) imaging has further demonstrated the potential of deep learning to maximize diagnostic performance while reducing subjectivity and errors induced by human interpretation.…”