Summarizing Chinese Medical Answer with Graph Convolution Networks and Question-focused Dual Attention

Zhang, Ningyu; Li, Juan; Chen, Xi; Zhang, Wei; Chen, Huajun

doi:10.18653/v1/2020.findings-emnlp.2

Cited by 19 publications

(9 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Abstractive Answer Summarization: Another line of work has attempted abstractive answer summarization by treating the tagged best answer as the gold summary of all the other answers (Chowdhury and Chakraborty, 2019;Chowdhury et al, 2020). Recent work summarizes answers to medical questions via a medical concept graph Zhang et al (2020) and incorporates multi-hop reasoning (Zhang et al, 2020) and answer relevance from a QA model into the summarization model (Su et al, 2021). Most related to our dataset creation, Chowdhury and Chakraborty (2019) present CQASumm, a dataset of about 100k automatically-created examples consisting of the best answer as the gold summary, which, however, contains noise due to automatic creation.…”

Section: Related Workmentioning

confidence: 99%

AnswerSumm: A Manually-Curated Dataset and Pipeline for Answer Summarization

Fabbri

Iyer

et al. 2022

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

Community Question Answering (CQA) fora such as Stack Overflow and Yahoo! Answers contain a rich resource of answers to a wide range of community-based questions. Each question thread can receive a large number of answers with different perspectives. One goal of answer summarization is to produce a summary that reflects the range of answer perspectives. A major obstacle for this task is the absence of a dataset to provide supervision for producing such summaries. Recent works propose heuristics to create such data, but these are often noisy and do not cover all answer perspectives present. This work introduces a novel dataset of 4,631 CQA threads for answer summarization curated by professional linguists. Our pipeline gathers annotations for all subtasks of answer summarization, including relevant answer sentence selection, grouping these sentences based on perspectives, summarizing each perspective, and producing an overall summary. We analyze and benchmark state-of-the-art models on these subtasks and introduce a novel unsupervised approach for multi-perspective data augmentation that boosts summarization performance according to automatic evaluation. Finally, we propose reinforcement learning rewards to improve factual consistency and answer coverage and analyze areas for improvement.

show abstract

Section: Related Workmentioning

confidence: 99%

AnswerSumm: A Manually-Curated Dataset and Pipeline for Answer Summarization

Fabbri

Iyer

et al. 2022

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

show abstract

“…• How to help models? We could use our approach of NAL to increase the weight of the knowledge learned in the pre-training task or leverage external knowledge (Zhang et al, 2019(Zhang et al, , 2020bYu et al, 2020;Zhang et al, 2020a).…”

Section: Case Studymentioning

confidence: 99%

ZJUKLAB at SemEval-2021 Task 4: Negative Augmentation with Language Model for Reading Comprehension of Abstract Meaning

Xie

Chen²,

Chen

et al. 2021

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021)

Self Cite

View full text Add to dashboard Cite

This paper presents our systems for the three Subtasks of SemEval Task4: Reading Comprehension of Abstract Meaning (ReCAM). We explain the algorithms used to learn our models and the process of tuning the algorithms and selecting the best model. Inspired by the similarity of the ReCAM task and the language pre-training, we propose a simple yet effective technology, namely, negative augmentation with language model. Evaluation results demonstrate the effectiveness of our proposed approach. Our models achieve the 4th rank on both official test sets of Subtask 1 and Subtask 2 with an accuracy of 87.9% and an accuracy of 92.8%, respectively 1 . We further conduct comprehensive model analysis and observe interesting error cases, which may promote future researches.

show abstract

“…(2) Large-scale KGs are required during both fine-tuning and inference for obtaining outputs of knowledge encoders. This incurs additional computation burden that limits their usage for real-world applications that require high inference speed (Zhang et al 2020).…”

Section: Introductionmentioning

confidence: 99%

DKPLM: Decomposable Knowledge-Enhanced Pre-trained Language Model for Natural Language Understanding

Zhang

Wang

et al. 2022

AAAI

View full text Add to dashboard Cite

Knowledge-Enhanced Pre-trained Language Models (KEPLMs) are pre-trained models with relation triples injecting from knowledge graphs to improve language understanding abilities.Experiments show that our model outperforms other KEPLMs significantly over zero-shot knowledge probing tasks and multiple knowledge-aware language understanding tasks. To guarantee effective knowledge injection, previous studies integrate models with knowledge encoders for representing knowledge retrieved from knowledge graphs. The operations for knowledge retrieval and encoding bring significant computational burdens, restricting the usage of such models in real-world applications that require high inference speed. In this paper, we propose a novel KEPLM named DKPLM that decomposes knowledge injection process of the pre-trained language models in pre-training, fine-tuning and inference stages, which facilitates the applications of KEPLMs in real-world scenarios. Specifically, we first detect knowledge-aware long-tail entities as the target for knowledge injection, enhancing the KEPLMs' semantic understanding abilities and avoiding injecting redundant information. The embeddings of long-tail entities are replaced by ``pseudo token representations'' formed by relevant knowledge triples. We further design the relational knowledge decoding task for pre-training to force the models to truly understand the injected knowledge by relation triple reconstruction. Experiments show that our model outperforms other KEPLMs significantly over zero-shot knowledge probing tasks and multiple knowledge-aware language understanding tasks. We further show that DKPLM has a higher inference speed than other competing models due to the decomposing mechanism.

show abstract

Summarizing Chinese Medical Answer with Graph Convolution Networks and Question-focused Dual Attention

Cited by 19 publications

References 21 publications

AnswerSumm: A Manually-Curated Dataset and Pipeline for Answer Summarization

AnswerSumm: A Manually-Curated Dataset and Pipeline for Answer Summarization

ZJUKLAB at SemEval-2021 Task 4: Negative Augmentation with Language Model for Reading Comprehension of Abstract Meaning

DKPLM: Decomposable Knowledge-Enhanced Pre-trained Language Model for Natural Language Understanding

Contact Info

Product

Resources

About