Task-Oriented Complex Ontology Alignment: Two Alignment Evaluation Sets

Abstract

Simple ontology alignments, largely studied, link one entity of a source ontology to one entity of a target ontology. One of the limitations of these alignments is, however, their lack of expressiveness which can be overcome by complex alignments. Although different complex matching approaches have emerged in the literature, there is a lack of complex reference alignments on which these approaches can be systematically evaluated. This paper proposes two sets of complex alignments between 10 pairs of ontologies from the well-known OAEI conference simple alignment dataset. The methodology for creating the alignment sets is described and takes into account the use of the alignments for two tasks: ontology merging and query rewriting. The ontology merging alignment set contains 313 correspondences and the query rewriting one 431. We report an evaluation of state-of-the art complex matchers on the proposed alignment sets.been carried out over the last fifteen years in the context of the Ontology Alignment Evaluation Campaigns (OAEI) 1 . Even though this well-known campaign proposes a task-oriented benchmark (the OA4QA track [28]), it does not propose a complex alignment benchmark.This paper proposes two alignment sets to extend the OAEI conference track dataset [3,36] with complex alignments for two task purposes: ontology merging and query rewriting. The methodology for creating the alignment sets is described and takes into account the use of the alignments for the two targeted tasks. Here we extend the work presented in [33] and in [31] by enriching the alignment sets with new pairs of ontologies and by considering the task for which the alignment is needed. We also extend the work in [31] and by adding an evaluation of three systems [23,24,13]. We extend the evaluation of the work in [33] by adding a new system described in [13] and by evaluating all the three systems on the ten pairs of ontologies for each alignment set.The paper is organised as follows. After giving the background on ontology matching ( §2) and discussing related work ( §3), we describe the methodology to create the alignments ( §4), the alignments themselves and their use for the evaluation of approaches ( §5). We conclude with a discussion on the proposal.

An Ontology Alignment Approach Combining Word Embedding and the Radius Measure

“…This decision is justified by the fact that our system was designed to support end-users by presenting a list of possible matches. We compared our matching results with the results of three stateof-the-art systems that were mentioned in [5]: Our system clearly outperforms the others on this benchmark, with a precision value equals to O.89 and recall value equals to 0.69 compared to 0.83, and 0.13 for the best state-of-the-art system. Many reasons can explain our result: (i) the cosine similarity between classes is much smaller, as a consequence this match gets discarded than the threshold (cosine similarity ('chair main', 'demo chair' = 0)).…”

“…This data set contains 57 correspondences made on five owl ontologies. Following the evaluation process presented in [5], we have taken into account only the alignments that exist in the complex data set and we ignored the alignment of simple data set. We assume that if our system is able to find the correct match between a proposed list, we consider that the entire proposed list is correct.…”

A consensual dataset for complex ontology matching evaluation

“…In particular, the complex Conference dataset results from a consensus between three raters manually generating the complex correspondences, with a special focus on the task of query rewriting. This consensual dataset extends the one presented in Thiéblin et al ., (2018b), where two (nonconsensual) alignment sets for two task purposes (ontology merging and query rewriting) were proposed.…”

“…We explore the issues the experts were faced with during the process and discuss the lessons learned and perspectives in the field. The contributions of this paper can be summarized as follows: we extend the methodology from Thiéblin et al ., (2018b) for constructing complex alignments, with a focus on the query-rewriting task. These guidelines can be adapted to the nature of the task or application.we present the consensual complex correspondence dataset that results from the adoption of the proposed methodology by three domain experts with the same level of expertise on the domain of conference organization.…”

“…we extend the methodology from Thiéblin et al ., (2018b) for constructing complex alignments, with a focus on the query-rewriting task. These guidelines can be adapted to the nature of the task or application.…”