Quality-based model for effective and robust multi-user pay-as-you-go ontology matching1

Abstract

Using a pay-as-you-go strategy, we allow for a community of users to validate or invalidate mappings obtained by an automatic ontology matching system using consensus for each mapping. The ultimate objectives are effectiveness-improving the quality of the obtained alignment (set of mappings) measured in terms of Fmeasure as a function of the number of user interactions-and robustness-making the system as much as possible impervious to user validation errors. Our strategy consists of two major steps: candidate mapping selection, which ranks mappings based on their perceived quality so that users are presented first with those mappings with lowest quality, and feedback propagation, which seeks to validate or invalidate those mappings that are perceived to be similar to the mappings already presented to the users for validation. The purpose of these two strategies is twofold: achieve greater improvements earlier and minimize overall user interaction. There are three important features of our approach: the use of a dynamic ranking mechanism to adapt to the new conditions after each user interaction, the presentation of each mapping for validation more than once-revalidation-because of possible user errors, and the immediate propagation of the user input on a mapping without first achieving consensus for that mapping. We study extensively the effectiveness and robustness of our approach as several of these parameters change, namely the error and revalidation rates, as a function of the number of iterations, to provide conclusive guidelines for the design and implementation of multiuser feedback ontology matching systems.

User validation in ontology alignment: functional assessment and impact

“…This means that AML does not extrapolate from the user feedback about a mapping to decide on the classification of multiple mapping candidates. While extrapolation (be it through active learning, feedback propagation, or other techniques) is an effective strategy for reducing user demand, it also implies that the system will be more heavily impacted by user errors (e.g., Cruz et al (2016)).…”

“…The user can adjust the size of the feedback propagation cluster via a threshold. The approach described in Cruz et al (2016) also discusses blocking propagation in the context of multi-user alignment. In this case, the feedback propagation can be controlled via the consensus of users for a given candidate mapping (SS.3.b).…”

Cross-lingual link discovery with TR-ESA

“…In relation to our cross-lingual interactive data linking approach, we want to incorporate methods that leverage links incrementally established by users to optimize the matching function in a pay-as-you-go fashion. We plan to adapt approaches proposed in the ontology matching and entity linking fields, which collect the feedback from individual or groups of users [7,5].…”

Ontology-based Instance Matching for Geospatial Urban Data Integration

“…Introducing human in the loop allows users to validate matching results and enables a system to automatically modify parameters (such as similarity thresholds) that are used in the matching process [3,8]. Together with the visualization process, analytical methods form the backbone of a visual analytics approach to ontology matching [9,20].…”