Multi-task Dynamic Transformer Network for Concurrent Bone Segmentation and Large-Scale Landmark Localization with Dental CBCT

Wang, Fan; Deng, Hannah H.; Wang, Li; Xiao, Deqiang; Kuang, Tianshu; Lin, Hung‐Ying; Gateño, Jaime; Shen, Steve Guofang; Yap, Pew-Thian; Xia, James J.; Shen, Dinggang

doi:10.1007/978-3-030-59719-1_78

Cited by 26 publications

(28 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This study, which followed best practice guidelines [20], is the first to train and test a deep learning segmentation model on such a large number of high-definition CMF CT scans. Our results are comparable or superior to those of previously published studies, despite the more challenging task we faced: all previous results were based on smaller test datasets (between 0 and 30 scans) and fewer segmentation masks [5,[8][9][10][11][12][13][14][15][16][17][18]. No previous work had included a cohort of consecutive patients, and only one previous publication had clearly stated that its database included patients with syndromic conditions [16].…”

Section: Discussionsupporting

confidence: 76%

“…In future works we plan to finetune this model in order to evaluate its performance with CBCT scans, another widespread and challenging imaging modality. Finally, we will attempt to use our segmentation results for automatic localization of anatomic landmarks, in order to provide cephalometric measurements for clinical diagnosis and treatment planning [12,16,17].…”

Section: Discussionmentioning

confidence: 99%

“…Deep learning algorithms might indeed be able to perform fully automatic segmentations [7]. Several authors have developed specific deep learning-based models for automatic segmentation of the upper skull, mandibular bone, teeth or mandibular canal [5,[8][9][10][11][12][13][14][15][16][17][18]. Most of these approaches relied on a U-Net convolutional architecture [19], and yielded promising results, with reported volumetric Dice Similarity Coefficient (vDSC) between 90% and 95%.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Fully automatic segmentation of craniomaxillofacial CT scans for computer-assisted orthognathic surgery planning using the nnU-Net framework

Dot

Schouman

Dubois

et al. 2021

Preprint

View full text Add to dashboard Cite

Objectives To evaluate the performance of the nnU-Net open-source deep learning framework for automatic multi-task segmentation of craniomaxillofacial (CMF) structures in CT scans obtained for computer-assisted orthognathic surgery. Methods Four hundred and fifty-three consecutive patients having undergone high-definition CT scans before orthognathic surgery were randomly distributed among a training/validation cohort (n = 300) and a testing cohort (n = 153). The ground truth segmentations were generated by 2 operators following an industry-certified procedure for use in computer-assisted surgical planning and personalized implant manufacturing. Model performance was assessed by comparing model predictions with ground truth segmentations. Examination of 45 CT scans by an industry expert provided additional evaluation. The model's generalizability was tested on a publicly available dataset of 10 CT scans with ground truth segmentations of the mandible. Results In the test cohort, mean volumetric Dice Similarity Coefficient (vDSC) & surface Dice Similarity Coefficient at 1mm (sDSC) were 0.96 & 0.97 for the upper skull, 0.94 & 0.98 for the mandible, 0.95 & 0.99 for the upper teeth, 0.94 & 0.99 for the lower teeth and 0.82 & 0.98 for the mandibular canal. Industry expert segmentation approval rates were 93% for the mandible, 89% for the mandibular canal, 82% for the upper skull, 69% for the upper teeth and 58% for the lower teeth. Conclusion While additional efforts are required for the segmentation of dental apices, our results demonstrated the model's reliability in terms of fully automatic segmentation of preoperative orthognathic CT scans.

show abstract

Section: Discussionsupporting

confidence: 76%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Fully automatic segmentation of craniomaxillofacial CT scans for computer-assisted orthognathic surgery planning using the nnU-Net framework

Dot

Schouman

Dubois

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Compared with the most advanced mandible segmentation methods, this method achieved better performance on craniofacial abnormalities and disease states; the DSC was 0.9382 on their own CBCT dataset and 0.9386 on the MICCAI Head and Neck Challenge 2015 dataset. Lian et al ( 92 ) introduced an efficient end-to-end deep network, the multi-task dynamic transformer network (DTNet), to perform concurrent mandible segmentation and large-scale landmark localization in one pass, for large-volume CBCT images. The network contributed to the quantitative analysis of craniomaxillofacial deformities.…”

Section: Clinical Application Of Automatic Image Segmentation In Stomatologymentioning

confidence: 99%

Deep Learning for Automatic Image Segmentation in Stomatology and Its Clinical Application

Luo¹,

Zeng²,

Chen³

et al. 2021

Front. Med. Technol.

View full text Add to dashboard Cite

Deep learning has become an active research topic in the field of medical image analysis. In particular, for the automatic segmentation of stomatological images, great advances have been made in segmentation performance. In this paper, we systematically reviewed the recent literature on segmentation methods for stomatological images based on deep learning, and their clinical applications. We categorized them into different tasks and analyze their advantages and disadvantages. The main categories that we explored were the data sources, backbone network, and task formulation. We categorized data sources into panoramic radiography, dental X-rays, cone-beam computed tomography, multi-slice spiral computed tomography, and methods based on intraoral scan images. For the backbone network, we distinguished methods based on convolutional neural networks from those based on transformers. We divided task formulations into semantic segmentation tasks and instance segmentation tasks. Toward the end of the paper, we discussed the challenges and provide several directions for further research on the automatic segmentation of stomatological images.

show abstract

“…In recent years, CNN-based joint segmentation and landmark detection (JSD) approaches [7,16] have been proposed to combine the two tasks via multi-task learning. In [7], the authors proposed a multi-task dynamic transformer network (DTNet) for concurrently segmenting mandible and detecting 64 landmarks.…”

Section: Introductionmentioning

confidence: 99%

SkullEngine: A Multi-stage CNN Framework for Collaborative CBCT Image Segmentation and Landmark Detection

Liu

Deng

Chen

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Accurate bone segmentation and landmark detection are two essential preparation tasks in computer-aided surgical planning for patients with craniomaxillofacial (CMF) deformities. Surgeons typically have to complete the two tasks manually, spending ∼12 hours for each set of CBCT or ∼5 hours for CT. To tackle these problems, we propose a multi-stage coarse-to-fine CNN-based framework, called SkullEngine, for high-resolution segmentation and large-scale landmark detection through a collaborative, integrated, and scalable JSD model and three segmentation and landmark detection refinement models. We evaluated our framework on a clinical dataset consisting of 170 CBCT/CT images for the task of segmenting 2 bones (midface and mandible) and detecting 175 clinically common landmarks on bones, teeth, and soft tissues. Experimental results show that SkullEngine significantly improves segmentation quality, especially in regions where the bone is thin. In addition, SkullEngine also efficiently and accurately detect all of the 175 landmarks. Both tasks were completed simultaneously within 3 minutes regardless of CBCT or CT with high segmentation quality. Currently, SkullEngine has been integrated into a clinical workflow to further evaluate its clinical efficiency.

show abstract

Multi-task Dynamic Transformer Network for Concurrent Bone Segmentation and Large-Scale Landmark Localization with Dental CBCT

Cited by 26 publications

References 19 publications

Fully automatic segmentation of craniomaxillofacial CT scans for computer-assisted orthognathic surgery planning using the nnU-Net framework

Fully automatic segmentation of craniomaxillofacial CT scans for computer-assisted orthognathic surgery planning using the nnU-Net framework

Deep Learning for Automatic Image Segmentation in Stomatology and Its Clinical Application

SkullEngine: A Multi-stage CNN Framework for Collaborative CBCT Image Segmentation and Landmark Detection

Contact Info

Product

Resources

About