Automatic image and text-based description for colorectal polyps using BASIC classification

Fonollà, Roger; Zander, Quirine E. W. van der; Schreuder, Ramon-Michel; Subramaniam, Sharmila; Bhandari, Pradeep; Masclee, Ad; Schoon, Erik J.; Sommen, Fons van der

doi:10.1016/j.artmed.2021.102178

Cited by 9 publications

(8 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the training set of the BERT module, only BLI images were used because this image enhancement mode is suitable for the application of BASIC. A more extensive explanation on CADTexD development and performance analyzed with technical metrics can be found in Fonollà et al (2021) [7]. CADTexD was tested by letting the algorithm generate textual descriptions as output for the same images of 55 CRPs in BLI that were optically diagnosed by the nineteen endoscopists.…”

Section: Cadtexd Detailsmentioning

confidence: 99%

“…In the training set of the BERT module, only BLI images were used because this image enhancement mode is suitable for the application of BA-SIC. A more extensive explanation of CADTexD development and performance analyzed with technical metrics can be found in Fonollà et al [7].…”

Section: Cadtexd Detailsmentioning

confidence: 99%

“…A CADx differentiating between benign (hyperplastic) and (pre)malignant (SSLs, adenomas, and CRCs) CRPs was previously developed successfully 6 . Using the fundamentals of that CADx, this study aimed to develop CADTexD (Computer-Aided Diagnosis with Textual Descriptions), a CADx capable of automatically generating textual descriptions of CRP features based on BASIC 7 .…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Automatic textual description of colorectal polyp features: explainable artificial intelligence

et al. 2023

Self Cite

View full text Add to dashboard Cite

Computer-aided diagnosis systems (CADx) can improve colorectal polyp (CRP) optical diagnosis. For integration into clinical practice, better understanding of artificial intelligence (AI) by endoscopists is needed. We aimed to develop an explainable AI CADx capable of automatically generating textual descriptions of CRPs. For training and testing of this CADx, textual descriptions of CRP size and features according to the Blue Light Imaging (BLI) Adenoma Serrated International Classification (BASIC) were used, describing CRP surface, pit pattern, and vessels. CADx was tested using BLI images of 55 CRPs. Reference descriptions with agreement by at least five out of six expert endoscopists were used as gold standard. CADx performance was analyzed by calculating agreement between the CADx generated descriptions and reference descriptions. CADx development for automatic textual description of CRP features succeeded. Gwet’s AC1 values comparing the reference and generated descriptions per CRP feature were: size 0.496, surface-mucus 0.930, surface-regularity 0.926, surface-depression 0.940, pits-features 0.921, pits-type 0.957, pits-distribution 0.167, and vessels 0.778. CADx performance differed per CRP feature and was particularly high for surface descriptors while size and pits-distribution description need improvement. Explainable AI can help comprehend reasoning behind CADx diagnoses and therefore facilitate integration into clinical practice and increase trust in AI.

show abstract

Section: Cadtexd Detailsmentioning

confidence: 99%

Section: Cadtexd Detailsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Automatic textual description of colorectal polyp features: explainable artificial intelligence

et al. 2023

Self Cite

View full text Add to dashboard Cite

show abstract

“…▶ E757 [11,12,13]. Despite these facts, computer-aided diagnosis of polyp morphology is a field yet to be explored: to the best of our knowledge, only one study describing assessment of polyp morphology by a computer system, as part of an algorithm for automated textual polyp image description, is available [75].…”

Section: Reviewmentioning

confidence: 99%

Advances in artificial intelligence and computer science for computer-aided diagnosis of colorectal polyps: current status

van Bokhorst,

Houwen,

Hazewinkel

et al. 2023

Endosc Int Open

View full text Add to dashboard Cite

“…Recently, the bidirectional encoder representations from transformers (BERT) have been developed for natural language processing (NLP), 16 which greatly improves the ability to recognize semantics and context and can generate medical reports. Fonollà et al 17 presented an AI-aided system that incorporated a BERT-based image captioning block to automatically describe colorectal polyps in colonoscopy. Xue et al 18 applied a recurrent generative model to a public data set to generate the imaging description paragraphs and impression sentences of CXR reports.…”

Section: Introductionmentioning

confidence: 99%

Comparison of Chest Radiograph Captions Based on Natural Language Processing vs Completed by Radiologists

et al. 2023

View full text Add to dashboard Cite

ImportanceArtificial intelligence (AI) can interpret abnormal signs in chest radiography (CXR) and generate captions, but a prospective study is needed to examine its practical value.ObjectiveTo prospectively compare natural language processing (NLP)-generated CXR captions and the diagnostic findings of radiologists.Design, Setting, and ParticipantsA multicenter diagnostic study was conducted. The training data set included CXR images and reports retrospectively collected from February 1, 2014, to February 28, 2018. The retrospective test data set included consecutive images and reports from April 1 to July 31, 2019. The prospective test data set included consecutive images and reports from May 1 to September 30, 2021.ExposuresA bidirectional encoder representation from a transformers model was used to extract language entities and relationships from unstructured CXR reports to establish 23 labels of abnormal signs to train convolutional neural networks. The participants in the prospective test group were randomly assigned to 1 of 3 different caption generation models: a normal template, NLP-generated captions, and rule-based captions based on convolutional neural networks. For each case, a resident drafted the report based on the randomly assigned captions and an experienced radiologist finalized the report blinded to the original captions. A total of 21 residents and 19 radiologists were involved.Main Outcomes and MeasuresTime to write reports based on different caption generation models.ResultsThe training data set consisted of 74 082 cases (39 254 [53.0%] women; mean [SD] age, 50.0 [17.1] years). In the retrospective (n = 8126; 4345 [53.5%] women; mean [SD] age, 47.9 [15.9] years) and prospective (n = 5091; 2416 [47.5%] women; mean [SD] age, 45.1 [15.6] years) test data sets, the mean (SD) area under the curve of abnormal signs was 0.87 (0.11) in the retrospective data set and 0.84 (0.09) in the prospective data set. The residents’ mean (SD) reporting time using the NLP-generated model was 283 (37) seconds—significantly shorter than the normal template (347 [58] seconds; P &lt; .001) and the rule-based model (296 [46] seconds; P &lt; .001). The NLP-generated captions showed the highest similarity to the final reports with a mean (SD) bilingual evaluation understudy score of 0.69 (0.24)—significantly higher than the normal template (0.37 [0.09]; P &lt; .001) and the rule-based model (0.57 [0.19]; P &lt; .001).Conclusions and RelevanceIn this diagnostic study of NLP-generated CXR captions, prior information provided by NLP was associated with greater efficiency in the reporting process, while maintaining good consistency with the findings of radiologists.

show abstract

Automatic image and text-based description for colorectal polyps using BASIC classification

Cited by 9 publications

References 33 publications

Automatic textual description of colorectal polyp features: explainable artificial intelligence

Automatic textual description of colorectal polyp features: explainable artificial intelligence

Advances in artificial intelligence and computer science for computer-aided diagnosis of colorectal polyps: current status

Comparison of Chest Radiograph Captions Based on Natural Language Processing vs Completed by Radiologists

Contact Info

Product

Resources

About