Predicting the impact of scientific concepts using full‐text features

Abstract

New scientific concepts, interpreted broadly, are continuously introduced in the literature, but relatively few concepts have a long‐term impact on society. The identification of such concepts is a challenging prediction task that would help multiple parties—including researchers and the general public—focus their attention within the vast scientific literature. In this paper we present a system that predicts the future impact of a scientific concept, represented as a technical term, based on the information available from recently published research articles. We analyze the usefulness of rich features derived from the full text of the articles through a variety of approaches, including rhetorical sentence analysis, information extraction, and time‐series analysis. The results from two large‐scale experiments with 3.8 million full‐text articles and 48 million metadata records support the conclusion that full‐text features are significantly more useful for prediction than metadata‐only features and that the most accurate predictions result from combining the metadata and full‐text features. Surprisingly, these results hold even when the metadata features are available for a much larger number of documents than are available for the full‐text features.

The evolutionary pattern of language in scientific writings: A case study of Philosophical Transactions of Royal Society (1665–1869)

“…Liu and Chen 2013;Ding et al 2014;Lu et al 2017), entity metrics (e.g. Ding et al 2013;Mckeown et al 2016), etc. Recently, linguistic complexity of scientific writing styles and scientific impacts (e.g.…”

Identifying problems and solutions in scientific text

“…2013 ) on 18,753,472 sentences from a biomedical corpus based on all full-text Pubmed articles (McKeown et al. 2016 ). From the 200 words which were semantically closest to “problem”, we manually selected 28 clear synonyms.…”

The impact of preprint servers in the formation of novel ideas

“…A number of studies propose different ideas to quantify originality in science and technology (Cozzens et al, 2010;Alexander et al, 2013;Rzhetsky et al, 2015;Rotolo et al, 2015;Wang and Chai, 2018;Shibayama and Wang, 2020) or their impact on the other works (Shi et al, 2010;Shahaf et al, 2012;Sinatra et al, 2016;Hutchins et al, 2016;Wesley-Smith et al, 2016;Herrmannova et al, 2018b,a;Zhao et al, 2019;Bornmann et al, 2019;Small et al, 2019). The prediction of breakthroughs, scientific impact and citation counts is a well developed area (Schubert and Schubert, 1997;Garfield et al, 2002;Dietz et al, 2007;Lokker et al, 2008;Shi et al, 2010;Uzzi et al, 2013;Alexander, 2013;Klimek et al, 2016;Tahamtan et al, 2016;McKeown et al, 2016;Clauset et al, 2017;Peoples et al, 2017;Salatino et al, 2018;Dong et al, 2018;Iacopini et al, 2018;Feldman et al, 2018;van den Besselaar and Sandström, 2018;Klavans et al, 2020). However, the question asked in these works is different from the one we ask.…”