Generating a Structured Summary of Numerous Academic Papers: Dataset and Method

Abstract

As time goes by, language evolves with word semantics changing. Unfortunately, traditional word embedding methods neglect the evolution of language and assume that word representations are static. Although contextualized word embedding models can capture the diverse representations of polysemous words, they ignore temporal information as well. To tackle the aforementioned challenges, we propose a graph-based dynamic word embedding (GDWE) model, which focuses on capturing the semantic drift of words continually. We introduce word-level knowledge graphs (WKGs) to store short-term and long-term knowledge. WKGs can provide rich structural information as supplement of lexical information, which help enhance the word embedding quality and capture semantic drift quickly. Theoretical analysis and extensive experiments validate the effectiveness of our GDWE on dynamic word embedding learning.

Hierarchical Catalogue Generation for Literature Review: A Benchmark

“…Dataset Multi-Xscience is proposed by Lu et al (2020), which focuses on writing the related work section of a paper based on its abstract with 4.4 articles cited in average. Dataset BigSurvey-MDS is the first large-scale multi-document scientific summarization dataset using review papers' introduction section as target (LIU et al, 2022), where previous work usually takes the section of related work as the target. Both BigSurvey and our HiCat-GLR task have more than 70 references, resulting in over 10,000 words of input, while their output is still the scale of a standard text paragraph, similar to Multi-Xscience.…”

“…It allows users to compose a template tree that consists of two types of nodes, dimension node and topic node. Shuaiqi et al (2022) trains classifiers based on BERT (Devlin et al, 2018) to conduct category-based alignment, where each sentence from academic papers is annotated into five categories: background, objective, method, result, and other. Next, each research topic's sentences are summarized and concatenated together.…”

“…However, these methods can only generate short summaries, while a literature review is required to provide a comprehensive and sufficient overview of a particular topic (Webster and Watson, 2002). Benefiting from the development of language modeling (Lewis et al, 2019), recent works directly attempt survey generation (Mohammad et al, 2009;Jha et al, 2015;Shuaiqi et al, 2022) but usually suffer from a disorganized generation without hierarchical guidance. As illustrated in Figure 1(A), we take gpt-3.5-turbo 1 , a large language model trained on massive amounts of diverse data including scientific papers, as the summarizer to generate scientific literature reviews.…”

Long Text and Multi-Table Summarization: Dataset and Method

“…Expect for the widely studied news summarization (Grusky et al, 2018;Fabbri et al, 2019), summarizing long documents received more attention in recent years. There are some datasets collected from different domains, including scientific literature (Cohan et al, 2018;Liu et al, 2022a), government reports (Huang et al, 2021), and books (Kryściński et al, 2021). The Financial Narrative Summarisation shared task in 2020 (El-Haj et al, 2020) delivered an annual report dataset from firms listed on the London Stock Exchange.…”

“…To model longer input sequences with limited GPU memory, Huang et al (2021) compare various efficient attention mechanisms for the encoder and propose an encoder-decoder attention named Hepos. Liu et al (2022a) identify and encode salient content in different aspects from diverse and long input content by category-based alignment and sparse attention mechanisms. Zhang et al (2022) divide the summarization process into multiple stages and keep segmenting, summarizing, and concatenating long inputs till they are compressed to a fixed length.…”