Deep learning-based automatic-bone-destruction-evaluation system using contextual information from other joints

Abstract

Background X-ray images are commonly used to assess the bone destruction of rheumatoid arthritis. The purpose of this study is to propose an automatic-bone-destruction-evaluation system fully utilizing deep neural networks (DNN). This system detects all target joints of the modified Sharp/van der Heijde score (SHS) from a hand X-ray image. It then classifies every target joint as intact (SHS = 0) or non-intact (SHS ≥ 1). Methods We used 226 hand X-ray images of 40 rheumatoid arthritis patients. As for detection, we used a DNN model called DeepLabCut. As for classification, we built four classification models that classify the detected joint as intact or non-intact. The first model classifies each joint independently, whereas the second model does it while comparing the same contralateral joint. The third model compares the same joint group (e.g., the proximal interphalangeal joints) of one hand and the fourth model compares the same joint group of both hands. We evaluated DeepLabCut’s detection performance and classification models’ performances. The classification models’ performances were compared to three orthopedic surgeons. Results Detection rates for all the target joints were 98.0% and 97.3% for erosion and joint space narrowing (JSN). Among the four classification models, the model that compares the same contralateral joint showed the best F-measure (0.70, 0.81) and area under the curve of the precision-recall curve (PR-AUC) (0.73, 0.85) regarding erosion and JSN. As for erosion, the F-measure and PR-AUC of this model were better than the best of the orthopedic surgeons. Conclusions The proposed system was useful. All the target joints were detected with high accuracy. The classification model that compared the same contralateral joint showed better performance than the orthopedic surgeons regarding erosion.

Detecting hand joint ankylosis and subluxation in radiographic images using deep learning: A step in the development of an automatic radiographic scoring system for joint destruction

“…In Table 3, we summarized related previous studies where the joint destruction in patients with RA was evaluated using X-ray images by artificial intelligence [25][26][27][28]. Miyama et al [25] developed a classification model for joint space narrowing (JSN) and erosion using VGG-16 in a small number of patients, in which the accuracy of the erosion classification was worse than that of JSN.…”

“…In Table 3, we summarized related previous studies where the joint destruction in patients with RA was evaluated using X-ray images by artificial intelligence [25][26][27][28]. Miyama et al [25] developed a classification model for joint space narrowing (JSN) and erosion using VGG-16 in a small number of patients, in which the accuracy of the erosion classification was worse than that of JSN. Ahalya et al [26] developed a classification model to determine RA from hand Xray images using GoogLeNet, in which only 10 epochs for pre-trained models and 50 epochs for customized CNN models were used.…”