Medical-VLBERT: Medical Visual Language BERT for COVID-19 CT Report Generation With Alternate Learning

Liu, Guangyi; Liao, Yinghong; Wang, Fuyu; Zhang, Bin; Zhang, Lu; Liang, Xiaodan; Wan, Xiang; Li, Shaolin; Li, Zhen; Zhang, Shuixing; Cui, Shuguang

doi:10.1109/tnnls.2021.3099165

Cited by 38 publications

(16 citation statements)

References 32 publications

(40 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…“TandemNet” is a dual attention model that can effectively combine image and text information, extract useful features, and focus attention for accurate image prediction. Medical Visual-Linguistic BERT, Medical-VLBERT, is another algorithm that has been used to generate medical reports of COVID-19 patients [ 67 ]. In this context, a curriculum learning framework, competence-based multimodal curriculum learning (CMCL), was proposed to solve the lack of generation of medical reports [ 82 ].…”

Section: Introduction Of the Explainable Ai Method: A Brief Overviewmentioning

confidence: 99%

Survey of Explainable AI Techniques in Healthcare

Chaddad

Peng

et al. 2023

Sensors

155

View full text Add to dashboard Cite

Artificial intelligence (AI) with deep learning models has been widely applied in numerous domains, including medical imaging and healthcare tasks. In the medical field, any judgment or decision is fraught with risk. A doctor will carefully judge whether a patient is sick before forming a reasonable explanation based on the patient’s symptoms and/or an examination. Therefore, to be a viable and accepted tool, AI needs to mimic human judgment and interpretation skills. Specifically, explainable AI (XAI) aims to explain the information behind the black-box model of deep learning that reveals how the decisions are made. This paper provides a survey of the most recent XAI techniques used in healthcare and related medical imaging applications. We summarize and categorize the XAI types, and highlight the algorithms used to increase interpretability in medical imaging topics. In addition, we focus on the challenging XAI problems in medical applications and provide guidelines to develop better interpretations of deep learning models using XAI concepts in medical image and text analysis. Furthermore, this survey provides future directions to guide developers and researchers for future prospective investigations on clinical topics, particularly on applications with medical imaging.

show abstract

Section: Introduction Of the Explainable Ai Method: A Brief Overviewmentioning

confidence: 99%

Survey of Explainable AI Techniques in Healthcare

Chaddad

Peng

et al. 2023

Sensors

155

View full text Add to dashboard Cite

show abstract

“…BERT [16] is a State-of-the-Art NLP model which has been extensively used in the healthcare domain [7,[24][25][26]. The self-attention mechanisms inherent to this model calculate the relationships between all pairs of tokens, allowing the model to identify the most informative words and phrases in each document.…”

Section: Bertmentioning

confidence: 99%

Building Large-Scale Registries from Unstructured Clinical Notes using a Low-Resource Natural Language Processing Pipeline

Tavabi

Pruneski

Golchin

et al. 2022

Preprint

View full text Add to dashboard Cite

Building clinical registries is an important step in improving the quality and safety of patient care. With the growing size of medical records, manual abstraction becomes more and more infeasible and impractical. On the other hand, Natural Language Processing Techniques have shown promising results in extracting valuable information from unstructured clinical notes. However, the structure and nature of clinical notes are very different from regular text that state-of-the- art NLP models are trained and tested on and they have their own set of challenges. In this study, we propose SE-K, an efficient and interpretable classification approach for extracting information from clinical notes, and show that it outperforms current state-of-the-art models in text classification. We use this approach to generate a 20- year comprehensive registry of anterior cruciate ligament reconstruction operations, one of the most common orthopedics operations among children and young adults. This registry can help us better understand the outcomes of this surgery and identify potential areas for improvement which can ultimately lead to better treatment outcomes.

show abstract

“…With the pandemic as a backdrop, there have already remained two main kinds of COVID-19 data research in visualization: micro-perspective in scientific visualization with this virus that is analyzed from a biomedical perspective and combined with clinical medicine (Nguyen et al 2021;Liu et al 2021). And macro-perspective in multiple related COVID-19 datasets, which contain such infection cases, recovery, and mortality rates with COVID and connected with social factors like geographical (Goetschel et al 2021), social media and journalism (Yu et al 2021;Leite et al 2020), trajectory of human mobility (Yang et al 2022), and other factors (Antweiler et al 2021;Gharizadeh et al 2020;Hua et al 2020).…”

Section: Visual Analytics Of Covid-19 Datamentioning

confidence: 99%

User-centered visual explorer of in-process comparison in spatiotemporal space

Dong

Ian

Liu

et al. 2022

J Vis

View full text Add to dashboard Cite

We propose a user-centered visual explorer (UcVE) for progressive comparing multiple visualization units in spatiotemporal space. We create unique unit visualization with the customizable aggregated view based on the visual metaphor of flower bursts. Each visualization unit is encoded with the abstraction of spatiotemporal properties. To reduce user cognition load, UcVE allows users to visualize, save, and track in-the-process exploration results. In coordination of storage sequence and block tracking views, UcVE can facilitate comparison with multiple visualization units concurrently, selected from historical and current exploration results. UcVE offers a flexible geo-based layout, with aggregation functions and temporal views of the timeline with categorized events, to maximize the user’s exploration capabilities. Finally, we demonstrate the usefulness by using COVID-19 datasets, case studies with different user scenarios, and expert feedback. Graphical abstract

show abstract

Medical-VLBERT: Medical Visual Language BERT for COVID-19 CT Report Generation With Alternate Learning

Cited by 38 publications

References 32 publications

Survey of Explainable AI Techniques in Healthcare

Survey of Explainable AI Techniques in Healthcare

Building Large-Scale Registries from Unstructured Clinical Notes using a Low-Resource Natural Language Processing Pipeline

User-centered visual explorer of in-process comparison in spatiotemporal space

Contact Info

Product

Resources

About