Classification of femur fracture in pelvic X-ray images using meta-learned deep neural network

Abstract

In the medical field, various studies using artificial intelligence (AI) techniques have been attempted. Numerous attempts have been made to diagnose and classify diseases using image data. However, different forms of fracture exist, and inaccurate results have been confirmed depending on condition at the time of imaging, which is problematic. To overcome this limitation, we present an encoderdecoder structured neural network that utilizes radiology reports as ancillary information at training. This is a type of meta-learning method used to generate sufficiently adequate features for classification. The proposed model learns representation for classification from X-ray images and radiology reports simultaneously. When using a dataset of only 459 cases for algorithm training, the model achieved a favorable performance in a test dataset containing 227 cases (classification accuracy of 86.78% and classification F1 score of 0.867 for fracture or normal classification). This finding demonstrates the potential for deep learning to improve performance and accelerate application of AI in clinical practice. In general, X-ray images and radiology reports offer complementary information to a physician who wants to make an informed decision. In the classical diagnosis process, the radiologist reads the image and notes the findings, and then the physician makes a corresponding diagnosis and appropriate decision. However, due to the success of deep learning, recent attempts to achieve a high-performance classifier with a deep neural network (DNN) that only inputs images have increased. Since GoogLeNet outperformed humans in 2014 1 , efforts to develop a high-performance classifier in various areas have continued. The deep learning method is currently popular; however, application to medical fields remains challenging. In particular, protection of patient medical information and unwillingness to share information between hospitals causes difficulty in acquiring a sufficient number of medical images to adequately train DNNs. This leads to performance degradation of DNNs with a relatively large number of parameters, thus requiring a more sophisticated learning algorithm. In addition, the numerous parameters require tuning based on physician assumptions and experience against concrete problems and training datasets, a tedious and resource-intensive task. Meta-learning is a recent technique to overcome (i.e., automate) this problem. The task is also known as "learning to learn" and aims to design models that can learn new tasks rapidly. Several studies have been proposed to apply metalearning techniques to medical images 2,3. Kim et al. 2 used few-shot learning, which is a type of meta-learning method for early diagnosis of glaucoma in fundus images. The authors developed a predictive model based on matching neural network architecture 4 , and showed that the model obtained greater effectiveness than vanilla DNNs. Maicas et al. 3 presented a simple experiment to demonstrate use of meta-learning for fine-tuning a medical image datas...

Look at here : Utilizing supervision to attend subtle key regions

“…On pelvic X-ray images, the AUC-ROC and F1-score obtained after applying Cut&Remain to ResNet-50 were improved by 6.6, and 6.4 for Normal-class classification, respectively. [17] presented a method for classifying femur fractures on X-ray images using deep learning trained with radiology reports. In the literature, they achieved an average F1 score of 81.7 in the 3-class classification task when using the whole images, which is not favorable to translate clinical practice.…”

Augmented Reality, Virtual Reality and Artificial Intelligence in Orthopedic Surgery: A Systematic Review

“…AI, a newly rising technology, achieved remarkable improvements in the medical field. Despite its almost flawless prediction accuracy in femur fracture classification cases [53] and general fracture detection [54], AI has been used the most for other purposes. Specifically, it has been used to grade the severity of lumbar spinal stenosis in epidemiological studies (31).…”

Deep Learning for Orthopedic Disease Based on Medical Image Analysis: Present and Future

“…For the hip, as with the shoulder, there has been an attempt to classify fractures by training the CNN model. Lee et al introduced a CNN model for training 786 anteroposterior pelvic plan radiographs using GoogLeNet-inception v3 [19]. The model classified a proximal femur fracture into type A (trochanteric region), type B (femur neck) and type C (femoral head) according to AO/OTA classification with an overall accuracy of 86.8%, showing a reasonable result.…”