A Framework to Build Games with a Purpose for Linked Data Refinement

Abstract

With the rise of linked data and knowledge graphs, the need becomes compelling to find suitable solutions to increase the coverage and correctness of datasets, to add missing knowledge and to identify and remove errors. Several approaches -mostly relying on machine learning and NLP techniques -have been proposed to address this refinement goal; they usually need a partial gold standard, i.e. some "ground truth" to train automatic models. Gold standards are manually constructed, either by involving domain experts or by adopting crowdsourcing and human computation solutions. In this paper, we present an open source software framework to build Games with a Purpose for linked data refinement, i.e. web applications to crowdsource partial ground truth, by motivating user participation through fun incentive. We detail the impact of this new resource by explaining the specific data linking "purposes" supported by the framework (creation, ranking and validation of links) and by defining the respective crowdsourcing tasks to achieve those goals. To show this resource's versatility, we describe a set of diverse applications that we built on top of it; to demonstrate its reusability and extensibility potential, we provide references to detailed documentation, including an entire tutorial which in a few hours guides new adopters to customize and adapt the framework to a new use case.

An Incremental Truth Inference Approach to Aggregate Crowdsourcing Contributions in Games with a Purpose

“…It is worth noting that we also released as open source the GWAP Enabler [23], a software framework to build Games with a Purpose which implements the incremental truth inference approach outlined in this paper. The two games mentioned in the evaluation section were developed on top of this framework.…”

Interplay of Game Incentives, Player Profiles and Task Difficulty in Games with a Purpose

“…The game adopts a repeated labeling approach [22] by asking different players to classify the same image; conversely, the same image is never given twice to the same player. Night Knights is built on top of our open source software framework for GWAPs [23].…”

To Incentivize or Not: Impact of Blockchain-Based Cryptoeconomic Tokens on Human Information Sharing Behavior

“…Thus, it has been suggested that information providers should structure their input in a contextualized way when sharing their data, utilizing semantic web technologies [15], such as linked data and ontologies [16,17], and evaluate the quality of information shared by other providers [18]. Nevertheless, as this would require additional effort on the part of the information providers [18], incentives such as gamification [19], reputation [20], money [21], or auctions [22] are suggested to motivate the data providers and thus improve the characteristics of the collected information. However, previous work on the incentivization of information sharing focuses on the quantity of collected information while excluding quality characteristics such as accuracy or contextualization [23].…”