From medical image to automatic medical report generation

Kisilev, Pavel; Walach, Eugene; Barkan, Ella; Ophir, Boaz; Alpert, Sharon; Hashoul, Sharbell Y.

doi:10.1147/jrd.2015.2393193

Cited by 33 publications

(9 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…tags, templates), rather than natural texts. Kisilev et al (2015) build a pipeline to predict the attributes of medical images. Shin et al (2016) adopt a CNN-RNN based framework to predict tags (e.g.…”

Section: Related Workmentioning

confidence: 99%

On the Automatic Generation of Medical Imaging Reports

Jing

Xie

Xing

2018

Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

326

329

View full text Add to dashboard Cite

Medical imaging is widely used in clinical practice for diagnosis and treatment. Report-writing can be error-prone for unexperienced physicians, and timeconsuming and tedious for experienced physicians. To address these issues, we study the automatic generation of medical imaging reports. This task presents several challenges. First, a complete report contains multiple heterogeneous forms of information, including findings and tags. Second, abnormal regions in medical images are difficult to identify. Third, the reports are typically long, containing multiple sentences. To cope with these challenges, we (1) build a multi-task learning framework which jointly performs the prediction of tags and the generation of paragraphs, (2) propose a co-attention mechanism to localize regions containing abnormalities and generate narrations for them, (3) develop a hierarchical LSTM model to generate long paragraphs. We demonstrate the effectiveness of the proposed methods on two publicly available datasets.

show abstract

Section: Related Workmentioning

confidence: 99%

On the Automatic Generation of Medical Imaging Reports

Jing

Xie

Xing

2018

Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

326

329

View full text Add to dashboard Cite

show abstract

“…The CNN models have matured and now demonstrate considerable performance guarantees through many developmental milestones (21)(22)(23)(24)(25)(26). Currently, research under the medical images domain widely leveraged the CNN models, including lesion detection, quantitative diagnosis, lesion segmentation, and so on (27)(28)(29)(30)(31).…”

Section: Introductionmentioning

confidence: 99%

A deep learning-based post-processing method for automated pulmonary lobe and airway trees segmentation using chest CT images in PET/CT

Xing¹,

Zhang²,

Nie³

et al. 2022

Quant Imaging Med Surg

View full text Add to dashboard Cite

Background: The proposed algorithm could support accurate localization of lung disease. To develop and validate an automated deep learning model combined with a post-processing algorithm to segment six pulmonary anatomical regions in chest computed tomography (CT) images acquired during positron emission tomography/computed tomography (PET/CT) scans. The pulmonary regions have five pulmonary lobes and airway trees. Methods: Patients who underwent both PET/CT imaging with an extra chest CT scan were retrospectively enrolled. The pulmonary segmentation of six regions in CT was performed via a convolutional neural network (CNN) of DenseVNet architecture with some post-processing algorithms. Three evaluation metrics were used to assess the performance of this method, which combined deep learning and the post-processing method. The agreement between the combined model and ground truth segmentations in the test set was analyzed.Results: A total of 640 cases were enrolled. The combined model, which involved deep learning and postprocessing methods, had a higher performance than the single deep learning model. In the test set, the alllobes overall Dice coefficient, Hausdorff distance, and Jaccard coefficient were 0.972, 12.025 mm, and 0.948, respectively. The airway-tree Dice coefficient, Hausdorff distance, and Jaccard coefficient were 0.849, 32.076 mm, and 0.815, respectively. A good agreement was observed between our segmentation in every plot. Conclusions:The proposed model combining two methods can automatically segment five pulmonary lobes and airway trees on chest CT imaging in PET/CT. The performance of the combined model was higher than the single deep learning model in each region in the test set.

show abstract

“…From the earliest LeNet (5) to AlexNet (6), VggNet (7), GoogleNet (8), ResNet (9), and the recent DenseNet (10), the performance of CNN models getting stronger and more mature. CNN models were widely used in medical images, include lesion detection, qualitative diagnosis, automatic generation of structured report, lesion extraction, organ delineation for radiotherapy, and so on (11)(12)(13)(14)(15).…”

Section: Introductionmentioning

confidence: 99%

Automatic segmentation of pulmonary lobes on low-dose computed tomography using deep learning

Zhang¹,

Ren²,

Tao³

et al. 2021

Ann Transl Med

View full text Add to dashboard Cite

Background: To develop and validate a fully automated deep learning-based segmentation algorithm to segment pulmonary lobe on low-dose computed tomography (LDCT) images.Methods: This study presents an automatic segmentation of pulmonary lobes using a fully convolutional neural network named dense V-network (DenseVNet) on lung cancer screening LDCT images. A total of 160 LDCT cases for lung cancer screening (100 in the training set, 10 in the validation set, and 50 in the test set) was included in this study. Specifically, the template of pulmonary lobes (the right lung consists of three lobes, and the left lung consists of two lobes) were obtained from pixel-level annotations by semiautomatic segmentation platform. Then, the model was trained under the supervision of the LDCT training set. Finally, the trained model was used to segment the LDCT in the test set. Dice coefficient, Jaccard coefficient, and Hausdorff distance were adopted as evaluation metrics to verify the performance of our segmentation model. Results:In this study, the model achieved the accurate segmentation of each pulmonary lobe in seconds without the intervention of researchers. The testing set consisted 50 LDCT cases were used to evaluate the performance of the segmentation model. The all-lobes Dice coefficient of the test set was 0.944, the Jaccard coefficient was 0.896, and the Hausdorff distance was 92.908 mm. Conclusions:The segmentation model based on LDCT can automatically and robustly and efficiently segment pulmonary lobes. It will provide effective location information and contour constraints for pulmonary nodule detection on LDCT images for lung cancer screening, which may have potential clinical application.

show abstract

From medical image to automatic medical report generation

Cited by 33 publications

References 8 publications

On the Automatic Generation of Medical Imaging Reports

On the Automatic Generation of Medical Imaging Reports

A deep learning-based post-processing method for automated pulmonary lobe and airway trees segmentation using chest CT images in PET/CT

Automatic segmentation of pulmonary lobes on low-dose computed tomography using deep learning

Contact Info

Product

Resources

About