Observing Linked Data Dynamics

Abstract

Abstract. In this paper, we present the design and first results of the Dynamic Linked Data Observatory: a long-term experiment to monitor the two-hop neighbourhood of a core set of eighty thousand diverse Linked Data documents on a weekly basis. We present the methodology used for sampling the URIs to monitor, retrieving the documents, and further crawling part of the two-hop neighbourhood. Having now run this experiment for six months, we analyse the dynamics of the monitored documents over the data collected thus far. We look at the estimated lifespan of the core documents, how often they go on-line or off-line, how often they change; we further investigate domain-level trends. Next we look at changes within the RDF content of the core documents across the weekly snapshots, examining the elements (i.e., triples, subjects, predicates, objects, classes) that are most frequently added or removed. Thereafter, we look at how the links between dereferenceable documents evolves over time in the two-hop neighbourhood.

Evaluating query and storage strategies for RDF archives

“…In parallel, a novel generation of semantically enhanced applications leverage this infrastructure to build services which can answer questions not possible before (thanks to the availability of SPARQL [21] which enables structured queries over this data). As previously reported [42,23], this published data is continuously undergoing changes (on a data and schema level). These changes naturally happen without a centralized monitoring nor pre-defined policy, following the scalefree nature of the Web.…”

Everything you always wanted to know about blank nodes

“…In a study of Linked Data dynamics, Käfer et al [41] applied a heuristic algorithm for guessing if two RDF graphs were equal (i.e., that two versions of an RDF graph remain the same): the algorithm sets the comparison of all pairs of blank nodes across documents as equal, where if the number of triples is the same and the set of RDF triples in both documents is equal under this heuristic, the algorithm considers the documents as provisionally isomorphic. If the heuristic returns true, only then is the Jena library used to execute a full isomorphism check.…”

Architecture of Linked Data Applications

“…Harth et al [22] introduce different research issues related to data crawling. Kafer et al [28] provide an empirical survey of the change frequency of Linked Data sources. Chapter ??…”