The Linked Data Benchmark Council Project

Abstract

Despite the fast growth and increasing popularity, the broad field of RDF and Graph database systems lacks an independent authority for developing benchmarks, and for neutrally assessing benchmark results through industry-strength auditing which would allow to quantify and compare the performance of existing and emerging systems.Inspired by the impact of the Transaction Processing Performance Council (TPC) Benchmarks on relational databases, the LDBC consortium formed by University and Industry researchers and practitioners has recently launched a European Commision sponsored project that will offer the first comprehensive set of open and vendor-independent benchmarks for RDF and Graph technologies. The consortium will incorporate the Linked Data Benchmark Council (LDBC) which will survive the project and will supervise the process of obtaining and reporting results as well as Supported by LDBC EU FP7 project A.Gubichev

Evaluating query and storage strategies for RDF archives

“…-While new retrieval features must be incorporated to benchmark archives, one should consider lessons learnt in previous recommendations on benchmarking RDF data management systems [1]. Although many benchmarks are defined for RDF stores [5,1] (see the Linked Data Benchmark Council project [7] for a general overview) and related areas such as relational databases (e.g. the well-known TPC 5 and recent TPC-H and TPC-C extensions to add temporal aspects to queries [24]) and graph databases [11], to the best of our knowledge, none of them are designed to address these particular considerations in 5 http://www.tpc.org/.…”

“…AnQL is a query language defined as a -relatively straightforward -extension of SPARQL, where a SPARQL triple pattern t is allowed to be annotated with a (temporal 7 ) label l as an annotated triple pattern of the form t : l. In our case, we assume for simplicity that the domain of annotations are simply (consecutive) version numbers, i.e. Moreover, for simplicity, we extend an AnQL BAP (basic annotated pattern), that is, a SPARQL Basic graph pattern (BGP) which may contain such annotated triple patterns as follows: Let P be a SPARQL graph pattern, then we write P : l as a syntactic short cut for an annotated pattern such that each triple pattern t ∈ P is replaced by t : l.…”

LOD-a-lot

“…Evaluation and benchmarking (UC2) have increasingly gained attention in the Semantic Web community [4]. However, Semantic Web evaluations still lack in terms of volume and variety.…”

Evaluating Query and Storage Strategies for RDF Archives

“…-While new retrieval features must be incorporated to benchmark archives, one should consider lessons learnt in previous recommendations on benchmarking RDF data management systems [1]. Although many benchmarks are defined for RDF stores [5,1] (see the Linked Data Benchmark Council project [7] for a general overview) and related areas such as relational databases (e.g. the well-known TPC 5 and recent TPC-H and TPC-C extensions to add temporal aspects to queries [24]) and graph databases [11], to the best of our knowledge, none of them are designed to address these particular considerations in RDF archiving.…”