Semantic and relational spaces in science of science: deep learning models for article vectorisation

Abstract

Over the last century, we observe a steady and exponential growth of scientific publications globally. The overwhelming amount of available literature makes a holistic analysis of the research within a field and between fields based on manual inspection impossible. Automatic techniques to support the process of literature review are required to find the epistemic and social patterns that are embedded in scientific publications. In computer sciences, new tools have been developed to deal with large volumes of data. In particular, deep learning techniques open the possibility of automated end-to-end models to project observations to a new, low-dimensional space where the most relevant information of each observation is highlighted. Using deep learning to build new representations of scientific publications is a growing but still emerging field of research. The aim of this paper is to discuss the potential and limits of deep learning for gathering insights about scientific research articles. We focus on document-level embeddings based on the semantic and relational aspects of articles, using Natural Language Processing (NLP) and Graph Neural Networks (GNNs). We explore the different outcomes generated by those techniques. Our results show that using NLP we can encode a semantic space of articles, while GNN we enable us to build a relational space where the social practices of a research community are also encoded.

An evolving international research collaboration network: spatial and thematic developments in co-authored higher education research, 1998–2018

“…Instead of selecting the journals based solely on author's judgement (Liu et al, 2019), we use the Content Analysis Toolkit for Academic Research (CATAR) 1 to apply agglomerative hierarchical clustering and multi-dimensional scaling based on bibliographical coupling similarity for cross-validation. Because the relatedness between scientific publications is multidimensional, no single measurement captures it perfectly; however, automated ways to analyze the semantic contents of articles are crucial as an effective response to the above-shown exponentially-increasing volume of scientific research articles (see Kozlowski et al, 2021). This method allowed us to double-check the topical relatedness of the overall 243 educational journals by analyzing the degree of overlap among the large volume of papers' references in all these educational journals (Small & Koenig, 1977;Tseng & Tsay, 2013;Yan & Ding, 2012).…”

Research on the spatiotemporal distribution and evolution of remote sensing: A data-driven analysis

“…Meanwhile, the number of remote sensing research related to in-depth learning increased from 3 in 2015 (accounting for 0.3% of the total number of research) to 98 in 2021 (5.43%), showing a significant upward trend (Figure 8). In recent years, with the soaring development of computer vision, such as image classification, target identification, and semantic segmentation, deep learning has been widely applied to remote sensing and has become an important innovation driver for remote sensing research (Zhang et al, 2016;Leung et al, 2017;Kozlowski et al, 2020).…”

Graph Embedding for Mapping Interdisciplinary Research Networks

“…This refers to the task of discovering useful representations of scientific articles, typically for use in downstream applications like classification, prediction, and retrieval [4]. These methods rely primarily on Natural Language Processing (NLP) and Network Analysis techniques to learn document embeddings according to a paper's content, a paper's citation relations, or some combination of these sources [12].…”

“…Representation learning for scientific documents is the task of representing research papers in some dense vector space, such that the important similarities and relations between the papers are preserved i.e., semantically related papers should have similar representations (or embeddings) [12]. Scientific articles may be related if they pertain to the same topic (i.e., have similar content), or if there is an application or transfer of knowledge from one to the other (i.e., a citation).…”