Semantic Web and Human Computation: The status of an emerging field

Abstract

This editorial paper introduces a special issue that solicited papers at the intersection of Semantic Web and Human Computation research. Research in that inter-disciplinary space dates back a decade, and has been acknowledged as a research line of its own by a seminal research manifesto published in 2015. But where do we stand in 2018? How did this research line evolve during the last decade? How do the papers in this special issue align with the main lines of work of the community? In this editorial we inspect and reflect on the evolution of research at the intersection of Semantic Web and Human Computation. We use a methodology based on Systematic Mapping Studies to collect quantitative bibliographic data which we analyze through the lens of research topics envisioned by the research manifesto to characterize the evolution of research in this area, thus providing a context for introducing the papers of this special issue. We found evidences of a thriving research field; while steadily maturing, the field offers a number of open research opportunities for work where Semantic Web best practices and techniques are applied to support and improve the state-of-the-art in Human Computation, but also for work that exploits the strength of both areas to address scientifically and societally relevant issues.

Deriving semantic validation rules from industrial standards: An OPC UA study

“…HC implies outsourcing specific tasks of a system, which cannot be fully automated, to human participants and leveraging the human processing power to solve those tasks. HC has been already successfully applied in a variety of domains for verification tasks [34] and we believe it is an important building block of a semi-automated method to support the steps needed for the derivation of (semi-)formal rules from pdf-based specifications. Next, the designed HC approach is explained and later in Section 7.5 a discussion of the results of an expert-sourcing validation campaign is presented.…”

Human-Centric Ontology Evaluation: Process and Tool Support

“…In parallel, the ontology should be prepared for the evaluation (data preparation) by extracting relevant ontological elements (13), which in [19] we accomplished using Apache Jena (jena.apache.org), designing batches (14) so that relevant tasks are grouped together, and extracting context (15) to be presented to the evaluators (e.g., in [19] pizza menu items are manually created) to be provided to the evaluator.…”

“…Indeed, a recent Systematic Mapping Study (SMS) in the field of human-centric evaluation of semantic resources identified 100 papers published on this topic in the last decade (2010-2020) [16]. A large portion of these papers (over 40%) relies on Human Computation and Crowdsourcing (HC&C) techniques [7,15], for example, to evaluate large biomedical ontologies [9] or to ensure the quality of Linked Data as a collaborative effort between experts and the crowd [1]. In [19], we applied HC&C for supporting the evaluation of P14 through Human Intelligence Tasks (HITs) such as those shown in Fig.…”

Verifying Extended Entity Relationship Diagrams with Open Tasks

“…The situation is similar in Knowledge Engineering where an increasing number of approaches focus on solving a similar problem of verifying domain models (Sabou et al 2018a). Domain (as opposed to system) models such as ontologies, taxonomies or knowledge graphs need to be verified for their correctness, as the following works exemplify.…”