ESBM: An Entity Summarization BenchMark

Abstract

Entity summarization is the problem of computing an optimal compact summary for an entity by selecting a size-constrained subset of triples from RDF data. Entity summarization supports a multiplicity of applications and has led to fruitful research. However, there is a lack of evaluation efforts that cover the broad spectrum of existing systems. One reason is a lack of benchmarks for evaluation. Some benchmarks are no longer available, while others are small and have limitations. In this paper, we create an Entity Summarization BenchMark (ESBM) which overcomes the limitations of existing benchmarks and meets standard desiderata for a benchmark. Using this largest available benchmark for evaluating general-purpose entity summarizers, we perform the most extensive experiment to date where 9 existing systems are compared. Considering that all of these systems are unsupervised, we also implement and evaluate a supervised learning based system for reference.

Incremental Entity Summarization With Formal Concept Analysis

“…In this section, we illustrate how to tokenize the objects of triples and construct the tokenized formal context using the following triples of the actual entity "3W AY F M " in ESBM dataset [34]:…”

“…The real-world dataset ESBM 1 we employed in experiments is available in [34], which contains two benchmark datasets for evaluating entity summarization. ESBM is currently the largest available benchmark dataset that can be found in the real-world.…”

“…On the basis of v1.0, ESBM v1.2 adds another 5 entities for each type of entity. We conducted the following three comparison experiments on ESBM v1.0 in terms of the efficiency, with the addition of a performance comparison 1. https://w3id.org/esbm 2. http://dbpedia.org/ 3. http://www.linkedmdb.org/ experiment on ESBM v1.0 and v1.2 compared to other stateof-the-art algorithms: Experiment I: First, we obtained the files of formal context using the Entity Summarization Benchmark datasets v1.0 and v1.2 [34]. After that, we converted the obtained files to adjacent matrices that are formal contexts of entities, as initial data in our experiments.…”

“…In this section, we will introduce the evaluation criteria that is adopted in [34], [36]. We utilize the following three indicators: F 1 − measure (so-called F1-score), M AP (Mean Average Precision), and N DCG (Normalized Discounted Cumulative Gain).…”

DeepLENS: Deep Learning for Entity Summarization

“…ESA encodes triples using graph embedding (TransE), and employs BiLSTM with supervised attention mechanism. Although it outperformed unsupervised methods, the improvement reported in [12] was rather marginal, around +7% compared with unsupervised FACES-E [4] on the ESBM benchmark [8]. It inspired us to explore more effective deep learning models for the task of general-purpose entity summarization.…”