Visual Analysis of Research Paper Collections Using Normalized Relative Compression

Abstract

The analysis of research paper collections is an interesting topic that can give insights on whether a research area is stalled in the same problems, or there is a great amount of novelty every year. Previous research has addressed similar tasks by the analysis of keywords or reference lists, with different degrees of human intervention. In this paper, we demonstrate how, with the use of Normalized Relative Compression, together with a set of automated data-processing tasks, we can successfully visually compare research articles and document collections. We also achieve very similar results with Normalized Conditional Compression that can be applied with a regular compressor. With our approach, we can group papers of different disciplines, analyze how a conference evolves throughout the different editions, or how the profile of a researcher changes through the time. We provide a set of tests that validate our technique, and show that it behaves better for these tasks than other techniques previously proposed.

An Empirical Evaluation of Document Embeddings and Similarity Metrics for Scientific Articles

“…Alvarez [7] use academic articles from PubMed, and they create a custom dataset for testing, where the similarity is obtained from common references. Finally, Vázquez created a synthetic testing dataset by taking research articles and iteratively adding pseudo-random sentences from one document to another [10]. This way, the documents become more and more similar.…”

“…Alvarez used Word Mover's Distance (WMD) [23], but only for titles and abstracts, due to its cost. We evaluate both cosine similarity, WMD, and other versions such as the Soft Cosine Similarity and the Relaxed Word Mover's Distance, and we also test a measure from Information Theory, Normalized Relative Compression [10].…”

“…Clearly, this is an entirely different distance calculation approach. As a normalized measure, it ranges from 0 to 1, and the result depends on the C(x y) factor, which can be: NRC is asymmetric and does not preserve the triangle inequality [10]. Thus, it is not a metric.…”

“…Thus, it is not a metric. Recent results obtained in its use for authorship attribution [36] and in the visual analysis of research papers [10] suggest that NRC is a good measure for many NLP tasks.…”

“…From a formal point, this behavior is what we would expect from a similarity measure that fulfills the Identity of indiscernibles and Symmetry axioms. Therefore, by iteratively repeating this process, similarity should increase monotonically in each iteration [10]. Note that, in most NLP tasks, this notion of fine-grain similarity is not needed.…”