Experiences from the anatomy track in the ontology alignment evaluation initiative

Abstract

BackgroundOne of the longest running tracks in the Ontology Alignment Evaluation Initiative is the Anatomy track which focuses on aligning two anatomy ontologies. The Anatomy track was started in 2005. In 2005 and 2006 the task in this track was to align the Foundational Model of Anatomy and the OpenGalen Anatomy Model. Since 2007 the ontologies used in the track are the Adult Mouse Anatomy and a part of the NCI Thesaurus. Since 2015 the data in the Anatomy track is also used in the Interactive track of the Ontology Alignment Evaluation Initiative.ResultsIn this paper we focus on the Anatomy track in the years 2007–2016 and the Anatomy part of the Interactive track in 2015–2016. We describe the data set and the changes it went through during the years as well as the challenges it poses for ontology alignment systems. Further, we give an overview of all systems that participated in the track and the techniques they have used. We discuss the performance results of the systems and summarize the general trends.ConclusionsAbout 50 systems have participated in the Anatomy track. Many different techniques were used. The most popular matching techniques are string-based strategies and structure-based techniques. Many systems also use auxiliary information. The quality of the alignment has increased for the best performing systems since the beginning of the track and more and more systems check the coherence of the proposed alignment and implement a repair strategy. Further, interacting with an oracle is beneficial.

Leveraging Structural Information in Ontology Matching

“…From the results we can see that current matching systems have all outpeformed baseline. Especially, in Group-1 (101-104) and Group-3 (221-247), most of the systems obtain very high performance and some of them even get 100% Fmeasure, such as RSDLWB [22] and XMap2 [22]. Therefore, it can be inferred that state of the art matching systems perform quite well on terminological matching since Group-1 and Group-3 did not suppress textual information in ontologies.…”

Exploiting Visual Similarities for Ontology Alignment

“…For analyzing the performance of the Visual Similarity ontology matching algorithm we ran it against the Ontology Alignment Evaluation Initiative (OAEI) Conference track of 2014 (Dragisic et al, 2014) 7 . The OAEI benchmarks are organized annually and have become a standard in ontology alignment tools evaluation.…”

“…For this test we again used the conference track benchmark dataset of OAEI 2014. For this dataset, results regarding the performance of the participating matching systems are published in OAEI's website and in (Dragisic et al, 2014). It can be seen from Table 1, in the line denoted with italic font, that the inclusion of the LexiVis ontology matching algorithm in the matching system results in better overall performance than running the system without it.…”

Fostering User Involvement in Ontology Alignment and Alignment Evaluation

“…In the absence of RAs, the evaluation of alignments requires exploration and comparison of multiple alignments. This involves users performing tasks at different levels of granularity [5,33,113] such as determining regions with similar or different number of mappings between the alignments, determining common or rarely found mappings and characterizing mappings as correct or incorrect. These activities serve as a basis to decide how good the obtained alignment is and thus to compare alignment tools and algorithms.…”

“…Papers I and II [65,32] focus on arguably the most cognitively intensive part of the alignment process-the manual validation of the candidate mappings, Paper III [64] addresses the issue of debugging the ontologies and their alignments and Paper IV [62] investigates the benefits from the latest technological advances to the alignment process. Papers V and VI [33,63] are connected to Research Question 2: How to efficiently support users during the evaluation of ontology alignments? and showcase the need of additional means, beyond precision and recall measures, for ontology alignment evaluation.…”