Benchmarking Matching Applications on the Semantic Web

Abstract

Abstract. The evaluation of matching applications is becoming a major issue in the semantic web and it requires a suitable methodological approach as well as appropriate benchmarks. In particular, in order to evaluate a matching application under different experimental conditions, it is crucial to provide a test dataset characterized by a controlled variety of different heterogeneities among data that rarely occurs in real data repositories. In this paper, we propose SWING (Semantic Web INstance Generation), a disciplined approach to the semi-automatic generation of benchmarks to be used for the evaluation of matching applications.

Evaluation of instance matching tools: The experience of OAEI

“…These phases will be briefly described in the Sections 4.2-4.4. These sections are a comprehensive summary of [34].…”

“…The benchmark is created using the SWING approach (Semantic Web INstance Generation) [34] a disciplined approach to the semi-automatic generation of benchmarks to be used for the evaluation of matching applications. The SWING approach has been implemented as a Java application and it is available at http://code.google.com/p/swing.…”

An unsupervised instance matcher for schema-free RDF data

“…Note that subject is counted among the Properties. Test cases 6 and 7 were artificially generated by SWING from underlying real-world IIMB data [71] ID Name Properties thus more appropriate to interpret this task as a link discovery task rather than the more specific instance matching task. Note also that this test case contains multilingual property values.…”

“…Test cases 6 and 7 are over the film domain and were artificially generated from real movie data using SWING, which injects controlled degrees of heterogeneity into an underlying corpus of real-world IIMB movie instances [71]. The types of heterogeneity (value, structural and semantic) were described in a companion paper [73], and the datasets were introduced as instance matching OAEI benchmarks in 2010 (along with Persons and Restaurants).…”

Populating a linked data entity name system

“…Test cases 6 and 7 are over the movies domain and were artificially generated from real movie data using SWING, which injects controlled degrees of heterogeneity into an underlying corpus of real-world IIMB movie instances (Ferrara, Montanelli, Noessner & Stuckenschmidt, 2011). The types of heterogeneity (value, structural and semantic) were earlier described in a companion paper (Ferrara, Lorusso, Montanelli &Varese, 2008), and the datasets were introduced as instance matching OAEI benchmarks in 2010 (along with…”