COVIDSum: A linguistically enriched SciBERT-based summarization model for COVID-19 scientific papers

Cai, Xiaoyan; Liu, Sen; Yang, Linlin; Lu, Yan; Zhao, Jintao; Shen, Dayong; Liu, Tianming

doi:10.1016/j.jbi.2022.103999

Cited by 26 publications

(29 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, given the strong ability of ChatGPT to extract key points and understand sentences, we can foresee potential promising results in text summarization. Specifically, ChatGPT might be valuable for domain-specific science paper summarization [86] and clinical report summarization [87]. Publicly available domain-specific science paper summarization datasets and clinical report datasets are rare and are often provided at small scales due to privacy concerns and the need for expert knowledge to generate annotated summaries.…”

Section: Conclusion and Discussionmentioning

confidence: 99%

AugGPT: Leveraging ChatGPT for Text Data Augmentation

Dai¹,

Liu²,

Liao³

et al. 2023

Preprint

View full text Add to dashboard Cite

Text data augmentation is an effective strategy for overcoming the challenge of limited sample sizes in many natural language processing (NLP) tasks. This challenge is especially prominent in the few-shot learning scenario, where the data in the target domain is generally much scarcer and of lowered quality. A natural and widely-used strategy to mitigate such challenges is to perform data augmentation on the training data to better capture the data invariance and increase the sample size. However, current text data augmentation methods either can not ensure the correct labeling of the generated data (lacking faithfulness) or can not ensure sufficient diversity in the generated data (lacking completeness), or both. Inspired by the recent success of large language models, especially the development of ChatGPT, which demonstrated improved language comprehension abilities, in this work, we propose a text data augmentation approach based on ChatGPT (named ChatAug). ChatGPT is trained on data with unparalleled linguistic richness and employs a reinforcement training process with large-scale human feedback, which endows the model with affinity to the naturalness of human language. Our text data augmentation approach ChatAug rephrases each sentence in the training samples into multiple conceptually similar but semantically different samples. The augmented samples can then be used in downstream model training. Experiment results on few-shot learning text classification tasks show the superior performance of the proposed ChatAug approach over state-of-the-art text data augmentation methods in terms of testing accuracy and distribution of the augmented samples.

show abstract

Section: Conclusion and Discussionmentioning

confidence: 99%

AugGPT: Leveraging ChatGPT for Text Data Augmentation

Dai¹,

Liu²,

Liao³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Since the majority of the preceding study evaluating transformer-based methods uses truncated documents [ 7 , 10 – 12 , 21 , 37 ], a corpus was also constructed for evaluating the proposed method by truncating long articles to 512 words long. In consideration that most of the salient knowledge of a research article is documented at the start of the research article, starting from the top of the article, we successively assimilate sentences to form one paragraph until 512 words long [ 1 ].…”

Section: Proposed Methodologymentioning

confidence: 99%

“…Globally, the SARS-CoV-2 virus had a destructive effect on communities since the upsurge of the COVID-19 pandemic in November 2019 [ 1 ]. Medical communities and researchers are under increased pressure to remain current with the articles due to the rapid growth of research articles [ 2 ].…”

Section: Introductionmentioning

confidence: 99%

“…There is an increase in the number of researchers participating in the advancement of diagnoses, therapies, and vaccines related to the pandemic. Furthermore, researchers publicize their current studies via scientific articles as scientific studies allow outcomes and visions to be shared across scientific associations [ 1 ]. Despite this, the manual generation of abstracts of published papers increases the workload of researchers and delays the writing and publication process.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

CovSumm: an unsupervised transformer-cum-graph-based hybrid document summarization model for CORD-19

Karotia

Susan

2023

J Supercomput

View full text Add to dashboard Cite

The number of research articles published on COVID-19 has dramatically increased since the outbreak of the pandemic in November 2019. This absurd rate of productivity in research articles leads to information overload. It has increasingly become urgent for researchers and medical associations to stay up to date on the latest COVID-19 studies. To address information overload in COVID-19 scientific literature, the study presents a novel hybrid model named CovSumm, an unsupervised graph-based hybrid approach for single-document summarization, that is evaluated on the CORD-19 dataset. We have tested the proposed methodology on the scientific papers in the database dated from January 1, 2021 to December 31, 2021, consisting of 840 documents in total. The proposed text summarization is a hybrid of two distinctive extractive approaches (1) GenCompareSum (transformer-based approach) and (2) TextRank (graph-based approach). The sum of scores generated by both methods is used to rank the sentences for generating the summary. On the CORD-19, the recall-oriented understudy for gisting evaluation (ROUGE) score metric is used to compare the performance of the CovSumm model with various state-of-the-art techniques. The proposed method achieved the highest scores of ROUGE-1: 40.14%, ROUGE-2: 13.25%, and ROUGE-L: 36.32%. The proposed hybrid approach shows improved performance on the CORD-19 dataset when compared to existing unsupervised text summarization methods.

show abstract

“…For example, the ClinicalRadioBERT is a specialized language model for radiation oncology [67]. Other applications include radiology report summarization [68], mental disorder detection [69], COVID-19 research summarization [70], and clinical information extraction [71]. However, the scale of existing language models is not comparable to recent developments such as ChatGPT and GPT-4.…”

Section: Applications Of Chatgpt/gpt-4mentioning

confidence: 99%

DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4

Liu¹,

Yu²,

Zhang³

et al. 2023

Preprint

View full text Add to dashboard Cite

The digitization of healthcare has facilitated the sharing and re-using of medical data but has also raised concerns about confidentiality and privacy. HIPAA (Health Insurance Portability and Accountability Act) mandates removing re-identifying information before the dissemination of medical records. Thus, effective and efficient solutions for de-identifying medical data, especially those in free-text forms, are highly needed. While various computer-assisted de-identification methods, including both rule-based and learning-based, have been developed and used in prior practice, such solutions still lack generalizability or need to be fine-tuned according to different scenarios, significantly imposing restrictions in wider use. The advancement of large language models (LLM), such as ChatGPT and GPT-4, have shown great potential in processing text data in the medical domain with zero-shot in-context learning, especially in the task of privacy protection, as these models can identify confidential information by their powerful named entity recognition (NER) capability. In this work, we developed a novel GPT4-enabled de-identification framework ("DeID-GPT") to automatically identify and remove the identifying information. Compared to existing commonly used medical text data de-identification methods, our developed DeID-GPT showed the highest accuracy and remarkable reliability in masking private information from the unstructured medical text while preserving the original structure and meaning of the text. This study is one of the earliest to utilize ChatGPT and GPT-4 for medical text data processing and de-identification, which provides insights for further research and solution development on the use of LLMs such as ChatGPT/GPT-4 in healthcare. Codes and benchmarking data information are available at https://github.com/yhydhx/ChatGPT-API.Recently, large language models (LLM), such as OpenAI's ChatGPT and GPT-4, have shown tremendous potential in analyzing and processing textual data, thus providing opportunities for downstream tasks in medical data analysis [10] [11]. ChatGPT shares a sibling relationship with InstructGPT. While InstructGPT is specifically designed to process prompts and provide comprehensive responses that adhere to pre-defined instructions, ChatGPT is developed to engage in conversation and generate contextual responses in a more naturalistic manner [12]. The latest released GPT-4 is capable of generating, editing, and collaborating with users on both creative and technical writing tasks, including

show abstract

COVIDSum: A linguistically enriched SciBERT-based summarization model for COVID-19 scientific papers

Cited by 26 publications

References 23 publications

AugGPT: Leveraging ChatGPT for Text Data Augmentation

AugGPT: Leveraging ChatGPT for Text Data Augmentation

CovSumm: an unsupervised transformer-cum-graph-based hybrid document summarization model for CORD-19

DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4

Contact Info

Product

Resources

About