Multitask Deep Learning for Segmentation and Classification of Primary Bone Tumors on Radiographs

Abstract

B one tumors include benign, intermediate, and malignant lesions, according to the classification system of the World Health Organization (1). Malignant neoplasms can be further divided into primary and secondary bone tumors or metastases (2,3). Radiography is the suggested primary imaging modality for the diagnosis of bone tumors because it can enable visualization of the location, destruction pattern, and periosteal reaction pattern of bone lesions (4,5). These destruction patterns reflect the biologic activity of bone lesions, through which they can be categorized as aggressive or nonaggressive (6). As demonstrated by Lodwick's well-established grading system, the destruction patterns of bone tumors observed on radiographs allow for evaluation of biologic activity and subsequently allow for risk assessment of malignancy (7,8). Primary bone tumors are uncommon. Thus, many radiologists may not be able to develop sufficient expertise to reliably identify and assess these lesions on radiographs (9). However, early detection and correct diagnosis are crucial for adequate and successful treatment (10). To improve the rates of early detection and correct assessment, an artificial intelligence model that could detect and accurately categorize bone lesions into malignant or benign bone lesions on radiographs may be beneficial. Recently, studies have shown that deep learning (DL) models reliably assess and detect a variety of diseases based on medical imaging data (11,12). Clinical implementation of these models may improve the reliability and accuracy of radiologic assessment, thus potentially leading to improved diagnostics and better patient outcomes (13). Recently, a preliminary study used DL to classify primary bone tumors on radiographs ( 14), but Background: An artificial intelligence model that assesses primary bone tumors on radiographs may assist in the diagnostic workflow.Purpose: To develop a multitask deep learning (DL) model for simultaneous bounding box placement, segmentation, and classification of primary bone tumors on radiographs. Materials and Methods:This retrospective study analyzed bone tumors on radiographs acquired prior to treatment and obtained from patient data from January 2000 to June 2020. Benign or malignant bone tumors were diagnosed in all patients by using the histopathologic findings as the reference standard. By using split-sample validation, 70% of the patients were assigned to the training set, 15% were assigned to the validation set, and 15% were assigned to the test set. The final performance was evaluated on an external test set by using geographic validation, with accuracy, sensitivity, specificity, and 95% CIs being used for classification, the intersection over union (IoU) being used for bounding box placements, and the Dice score being used for segmentations. Results:Radiographs from 934 patients (mean age, 33 years 6 19 [standard deviation]; 419 women) were evaluated in the internal data set, which included 667 benign bone tumors and 267 malignant bone tumors. Six hundred f...

Applications of machine learning for imaging-driven diagnosis of musculoskeletal malignancies—a scoping review

“…Yet another indication is that problem statements of most studies do not reflect real clinical scenarios. Most studies aim at distinguishing two to three specific tumour entities [10,16,34,43,[46][47][48] or assessing tumour malignancy [15,18,19,22,26,28,32,33,35,36,39,40,42,44,45]. If one fed a third entity to a two-entity classifier, the model would try to fit the third entity into one of the first two entity classes.…”

“…While confining a tumour entity from another is an imperative step in tumour assessment, nonetheless, most sarcoma diagnoses are incidental findings, and in daily practice, MSK radiologists and orthopaedic surgeons are first confronted with detecting a potential sarcoma at all [1,4,53]. Whereas von Schacky et al [42] aimed at differentiating various tumour entities, thus modelling a more realistic clinical scenario, the results were only moderate. More general models are needed to comply with clinical needs and difficulties.…”

“…This approach is similar to the diagnostic procedure a radiologist would use to diagnose MSK lesions. A second noticeable study was published by von Schacky et al [42]: they presented a multi-task DL model that shows the potential of state-of-the-art DL by simultaneously detecting, segmenting and classifying image data.…”

Multi-task Deep Learning for Cerebrovascular Disease Classification and MRI-to-PET Translation

“…Recently, multi-task neural networks have shown superior performance to other individual neural network architectures on different medical imaging applications [16], [17]. This type of neural networks simultaneously integrates different pieces of information from diverse tasks to improve the overall performance of the network and leads to better generalization under real-life conditions [18].…”

“…This model showed impressive results with an AUC score greater than 97% for the diagnosis task and a Dice score of 0.88 for the segmentation task. Similarly, Von et al in [17] developed a multi-task deep learning model for the classification and segmentation of primary bone tumors on musculoskeletal radiographs. This model was able to distinguish between malignant and benign tumors with an average classification accuracy of 80.2% and segment the bone lesions with an average Dice coefficient of 0.60.…”