How Ontologies are Made: Studying the Hidden Social Dynamics Behind Collaborative Ontology Engineering Projects

Abstract

Traditionally, evaluation methods in the field of semantic technologies have focused on the end result of ontology engineering efforts, mainly, on evaluating ontologies and their corresponding qualities and characteristics. This focus has led to the development of a whole arsenal of ontologyevaluation techniques that investigate the quality of ontologies as a product. In this paper, we aim to shed light on the process of ontology engineering construction by introducing and applying a set of measures to analyze hidden social dynamics. We argue that especially for ontologies which are constructed collaboratively, understanding the social processes that have led to its construction is critical not only in understanding but consequently also in evaluating the ontology. With the work presented in this paper, we aim to expose the texture of collaborative ontology engineering processes that is otherwise left invisible. Using historical change-log data, we unveil qualitative differences and commonalities between different collaborative ontology engineering projects. Explaining and understanding these differences will help us to better comprehend the role and importance of social factors in collaborative ontology engineering projects. We hope that our analysis will spur a new line of evaluation techniques that view ontologies not as the static result of deliberations among domain experts, but as a dynamic, collaborative and iterative process that needs to be understood, evaluated and managed in itself. We believe that advances in this direction would help our community to expand the existing arsenal of ontology evaluation techniques towards more holistic approaches.

Discovering Beaten Paths in Collaborative Ontology-Engineering Projects using Markov Chains

“…Strohmaier et al [23] investigated the hidden social dynamics that take place in collaborative ontology-engineering projects from the biomedical domain and provides new metrics to quantify various aspects of the collaborative engineering processes. Wang et al [24] have used association-rule mining to analyze user editing patterns in collaborative ontologyengineering projects.…”

Analysis and Prediction of User Editing Patterns in Ontology Development Projects

“…We and other investigators have used change data from ontologies to measure the level of community activities in biomedical ontologies [14], to migrate data from an old version of an ontology to a new one [12], and to analyze user roles in the process of collaboration [6,7,23,26]. For example, we have demonstrated that we can use the change data to assess the level of stabilization in ontology content [26], to find implicit user roles [7], and to describe the collaboration qualitatively [23]. For example, we found that changes to ICD-11 tend to propagate along the class hierarchy: A user who alters a property value for a class is significantly more likely to make a change to a property value for a subclass of that class than to make an edit anywhere else in the ontology [19].…”

The protégé project

“…Protégé at the beginning was viewed merely as a means to an end. More recent grants, however, have supported our original research to study collaborative ontology development (Tudorache et al, 2011; Strohmaier et al, 2013), methods for ontology alignment (Noy and Musen, 2003; Ghazvinian et al, 2009), ontology evaluation through crowdsourcing (Mortensen et al, 2014), ontology visualization (Storey et al, 2001), and methods for the very early stages of ontology conceptualization (Zhang et al, 2015). These academic pursuits are what make our laboratory an exciting place for students and post-docs, and they are what keep the research funding flowing.…”