Explaining Relationships Between Scientific Documents

Abstract

We address the task of explaining relationships between two scientific documents using natural language text. This task requires modeling the complex content of long technical documents, deducing a relationship between these documents, and expressing that relationship in text. Successful solutions can help improve researcher efficiency in search and review. In this paper, we operationalize this task by using citing sentences as a proxy. We establish a large dataset for our task. We pretrain a large language model to serve as the foundation for autoregressive approaches to the task. We explore the impact of taking different views on the two documents, including the use of dense representations extracted with scientific information extraction systems. We provide extensive automatic and human evaluations which show the promise of such models, and make clear the challenges for future work.

Automatic Related Work Generation: A Meta Study

“…In parallel to Ge et al [37] and Chen et al [18]: Luu et al [75] frame their task as using citation sentence as partial supervision for explaining relationships between two scientific documents. They use citation sentences in the S2ORC dataset [73] with a single reference that links back to the corpus as their dataset.…”

“…Compared to the systems proposed above, Luu et al [75] propose a simpler approach by further fine-tuning an existing model, which is easier to apply to future scenarios. However, the GPT2 [94] base model only has a limited context window of 512 (base) or 1024 (large), which is a severe bottleneck for modeling the relationship between two documents.…”

“…As a result, most of the current abstractive related work generation works limit their scope to generating citation sentences, which are the most challenging part of related work generation. These works may even not mention "related work generation" as their task, but instead, phrase it as "citation text generation" [37,117] or "explaining relationships between scientific documents" [75], but they are essentially a fundamental component of related work generation task.…”

“…Inputs Target AbuRa'ed et al [2] Cited title, abstract Citation sentence w/ single reference Xing et al [117] Context sentences, Citation sentence w/ single single cited abstract reference Ge et al [37] Citation network, single cited Citation sentence, citation abstract, context sentences function, salient sentence in cited abstracts Luu et al [75] Intro of the citing paper, Citation sentence w/ single reference named entities of the cited papers Chen et al [18] Cited abstracts A paragraph w/ 2+ citations As a pioneering work abstractive citation sentence generation, AbuRa'ed et al [2] work on the abstractive generation of citation sentences given the concatenation of cited paper's title and abstract. In parallel, Xing et al [117]'s model takes 3 sentences each before and after the target citation sentence, and the cited paper's abstract as the inputs.…”

“…Although the problem of automatic related work generation was proposed over 10 years ago, this problem received little attention until very recently, when a few works revisited this problem with neural network-based systems [2,18,25,37,75,117]. There are only a few existing papers that explicitly study related work generation.…”