Summarization of COVID-19 news documents deep learning-based using transformer architecture

Abstract

Facing the news on the internet about the spreading of Corona virus disease 2019 (COVID-19) is challenging because it is required a long time to get valuable information from the news. Deep learning has a significant impact on NLP research. However, the deep learning models used in several studies, especially in document summary, still have a deficiency. For example, the maximum output of long text provides incorrectly. The other results are redundant, or the characters repeatedly appeared so that the resulting sentences were less organized, and the recall value obtained was low. This study aims to summarize using a deep learning model implemented to COVID-19 news documents. We proposed transformer as base language models with architectural modification as the basis for designing the model to improve results significantly in document summarization. We make a transformer-based architecture model with encoder and decoder that can be done several times repeatedly and make a comparison of layer modifications based on scoring. From the resulting experiment used, ROUGE-1 and ROUGE-2 show the good performance for the proposed model with scores 0.58 and 0.42, respectively, with a training time of 11438 seconds. The model proposed was evidently effective in improving result performance in abstractive document summarization.

Natural Language Processing-based Summary Algorithm for Understanding Online News

“…For training, validation, and testing, the dataset is divided into three sections in predetermined ratios. To help manage the language required for model training, contractions and printable checks are provided (Hayatin et al, 2021).…”

Improving Measurement of Effectiveness of Blended Learning in Iraqi Education Using SVM

“…As a result, many universities now employ online learning as a viable option. There have been many studies dealing with e-learning [12], including:…”

Improving Coverage and Novelty of Abstractive Text Summarization Using Transfer Learning and Divide and Conquer Approaches

“…Automatic Text Summarization (ATS) is one of the most valuable systems that benefit humanity. It is utilized in various fields, including medicine [1], business [2], and education. Current research progress in leveraging State-ofthe-Art (SotA) pre-trained language models demonstrates impressive improvements in both extractive and abstractive types of ATS.…”

“…Abstractive ATS datasets vary based on the length of the input documents. They can be categorized into shortlength documents (DUC 1 and Gigaword [3], [4]), medium-length documents (CNN/DM [5], [6], XSUM [7], and Reddit_TIFU [8]), and long-length documents (arXiv and PubMed [9]). In this study, we will focus on increasing the coverage of the summary by utilizing the output design of each dataset.…”