Deep learning-based automatic-bone-destruction-evaluation system using contextual information from other joints

Abstract

Background: X-ray images are commonly used for assessing presence of bone destruction of rheumatoid arthritis. The purpose of this study is to propose an automatic-bone-destruction-evaluation system fully utilizing deep neural networks (DNN). This system detects all target joints of the modified Sharp/van der Heijde score (SHS) from a hand X-ray image. It then classifies every target joint as intact (SHS = 0) or non-intact (SHS ≥ 1).Methods: We used 226 hand X-ray images of 40 rheumatoid arthritis patients. As for detection, we used a DNN model called DeepLabCut. As for classification, we built four classification models that classify the detected joint as intact or non-intact. First, each joint is independently classified; second, the same contralateral joint is compared; third, the same joint group (e.g., the proximal interphalangeal joints) of one hand are compared; fourth, the same joint group of both hands are compared. We evaluated DeepLabCut’s detection performance and classification models’ performances. The classification models’ performances were compared to three orthopedic surgeons.Results: Detection rates for all the target joints were 98.0% and 97.3% for erosion and joint space narrowing (JSN). Among the four classification models, the model that compares the same contralateral joint showed the best F-measure (0.70, 0.81) and area under the curve of the precision-recall curve (PR-AUC) (0.73, 0.85) regarding erosion and JSN. As for erosion, the F-measure and PR-AUC of this model were better than the best of the orthopedic surgeons.Conclusions: The proposed system was useful. All the target joints were detected with high accuracy. The classification model that compared the same contralateral joint showed better performance than the orthopedic surgeons regarding erosion.