Graph Enhanced Contrastive Learning for Radiology Findings Summarization

Hu, Jinpeng; Li, Zhuo; Chen, Zhihong; Li, Zhen; Xiang, Wei; Chang, Tsung-Hui

doi:10.48550/arxiv.2204.00203

Cited by 3 publications

(3 citation statements)

References 29 publications

(51 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…General Impression generation can be regarded as a special type of summarization task in the medical domain, aiming to summarize findings and generate impressions. There are many methods proposed for this area (Gharebagh et al, 2020;Hu et al, 2021;Hu et al, 2022;Karn et al, 2022;MacAvaney et al, 2019;Zhang et al, 2020c). MacAvaney et al (2019);Gharebagh et al (2020) proposed to extract medical ontologies and then utilize a separate encoder to extract features from such critical words for improving the decoding process and thus promote AIG.…”

Section: Radiology Impression Generationmentioning

confidence: 99%

Improving Radiology Summarization with Radiograph and Anatomy Prompts

Hu¹,

Chen²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

The impression is crucial for the referring physicians to grasp key information since it is concluded from the findings and reasoning of radiologists. To alleviate the workload of radiologists and reduce repetitive human labor in impression writing, many researchers have focused on automatic impression generation. However, recent works on this task mainly summarize the corresponding findings and pay less attention to the radiology images. In clinical, radiographs can provide more detailed valuable observations to enhance radiologists' impression writing, especially for complicated cases. Besides, each sentence in findings usually focuses on single anatomy, so they only need to be matched to corresponding anatomical regions instead of the whole image, which is beneficial for textual and visual features alignment. Therefore, we propose a novel anatomy-enhanced multimodal model to promote impression generation. In detail, we first construct a set of rules to extract anatomies and put these prompts into each sentence to highlight anatomy characteristics. Then, two separate encoders are applied to extract features from the radiograph and findings. Afterward, we utilize a contrastive learning module to align these two representations at the overall level and use a co-attention to fuse them at the sentence level with the help of anatomyenhanced sentence representation. Finally, the decoder takes the fused information as the input to generate impressions. The experimental results on two benchmark datasets confirm the effectiveness of the proposed method, which achieves state-of-the-art results.

show abstract

Section: Radiology Impression Generationmentioning

confidence: 99%

Improving Radiology Summarization with Radiograph and Anatomy Prompts

Hu¹,

Chen²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…To improve the robustness of the graph structure, refs. [ 26 , 27 , 28 , 29 , 30 ] introduced contrast learning in the model for pulling positive samples close to push away negative samples. Most of the encoders of existing models are based on sentences or words with low reliability, and the joint summary model proposed in [ 31 ] improves the performance of summary generation.…”

Section: Introductionmentioning

confidence: 99%

Text Summarization Method Based on Gated Attention Graph Neural Network

Huang

et al. 2023

Sensors

View full text Add to dashboard Cite

Text summarization is an information compression technology to extract important information from long text, which has become a challenging research direction in the field of natural language processing. At present, the text summary model based on deep learning has shown good results, but how to more effectively model the relationship between words, more accurately extract feature information and eliminate redundant information is still a problem of concern. This paper proposes a graph neural network model GA-GNN based on gated attention, which effectively improves the accuracy and readability of text summarization. First, the words are encoded using a concatenated sentence encoder to generate a deeper vector containing local and global semantic information. Secondly, the ability to extract key information features is improved by using gated attention units to eliminate local irrelevant information. Finally, the loss function is optimized from the three aspects of contrastive learning, confidence calculation of important sentences, and graph feature extraction to improve the robustness of the model. Experimental validation was conducted on a CNN/Daily Mail dataset and MR dataset, and the results showed that the model in this paper outperformed existing methods.

show abstract

“…Moreover, we design a feature-filtering gate that can better fuse and filter the information of multiple expert modules. (2) To solve the second problem, we propose a solution inspired by a recent contrastive learning model [13]. Our model applies contrastive learning to English hate speech detection by using a swearing dictionary and an identity term dictionary to construct positive and negative examples.…”

Section: Introductionmentioning

confidence: 99%

Keyword-Enhanced Multi-Expert Framework for Hate Speech Detection

Zhong

et al. 2022

Mathematics

View full text Add to dashboard Cite

The proliferation of hate speech on the Internet is harmful to the psychological health of individuals and society. Thus, establishing and supporting the development of hate speech detection and deploying evasion techniques is a vital task. However, existing hate speech detection methods tend to ignore the sentiment features of target sentences and have difficulty identifying some implicit types of hate speech. The performance of hate speech detection can be significantly improved by gathering more sentiment features from various sources. In the use of external sentiment information, the key information of the sentences cannot be ignored. Thus, this paper proposes a keyword-enhanced multiexperts framework. To begin, the multi-expert module of multi-task learning is utilized to share parameters and thereby introduce sentiment information. In addition, the critical features of the sentences are highlighted by contrastive learning. This model focuses on both the key information of the sentence and the external sentiment information. The final experimental results on three public datasets demonstrate the effectiveness of the proposed model.

show abstract

Graph Enhanced Contrastive Learning for Radiology Findings Summarization

Cited by 3 publications

References 29 publications

Improving Radiology Summarization with Radiograph and Anatomy Prompts

Improving Radiology Summarization with Radiograph and Anatomy Prompts

Text Summarization Method Based on Gated Attention Graph Neural Network

Keyword-Enhanced Multi-Expert Framework for Hate Speech Detection

Contact Info

Product

Resources

About