Evaluation and Refinement of Emergency Situation Ontology

Abstract

Ontologies provide a machine-processable description of entities and their properties, relationships and constraints, thus they can depict the semantics of disaster situations and related emergency tasks and thereby help to create connections between them for an efficient emergency response. As an ontology acts as the basic structure and knowledge base of an application, evaluation and assessment of the ontology are a critical point of the development process. Through the evaluation, the quality and the content of the ontology is assessed and it ensures that the ontology is well built, structured and contains all important concepts and relationships between them for sufficient reasoning. In this paper, an evaluation framework is proposed to evaluate an emergency situation ontology for which existing evaluation methods have been combined into a single framework, dividing the methods used into two phases: verification and validation. The verification of the ontology ensures that the ontology is correctly built. It evaluates the structure, functionality and representation of the ontology. Different metrics and common pitfalls are used to detect errors. The validation of the ontology ensures that the right ontology for the given application is built. This is achieved by competency questions and expert interviews.

Ontology: Components and Evaluation, a Review

“…The verification has the next focus: -Lexical: the lexical verification is related to the vocabulary used for conceptualizing the domain and is regularly made with precision, recall and F -measure [31] [3]. This verification includes aspects related to the reusability [15]. -Taxonomic: specifically, this verification is only focused on is-a and has-a relationships within the ontology.…”

“…-Taxonomic: specifically, this verification is only focused on is-a and has-a relationships within the ontology. -Semantic: is based on consistency by the semantic features of the ontology [37] considering the meaning and content of these features [15]. -Context: can be evaluated by other web ontologies or specific applications [28] which are in the same domain.…”

“…-Syntactic: this verification is about the coherence in the ontology definitions [28]. For a deep verification, it is necessary using some criteria [15]. -Structural, architectural and design: have the purpose to identify the absence of the main concepts of a domain in the structure, loops, concepts with the same definition but different name, among other errors.…”

“…-Data driven: for applying this validation is necessary to compare the information of the ontology with existing data about the same domain meanly using a corpus of text documents [3]. A very important aspect in this validation is determining if the ontology has enough elements for representing the domain [15]. -User based: basically, this validation is about the experience and perspective around the final user because it becomes suggestive and empiric; for minimizing the error it is advisable the user be an expert in the ontology domain [17].…”

“…How the ontology responses in future uses [20] Clarity To communicate about the meaning of the terms independently of the context [20] Closeness Index Measures the closeness or structural resemblance among the concepts [14] Similarity Index About the proposition correctness into knowledge structure [14] Accuracy Measures the representation of knowledge within the ontology in relation to the real world [37] Coherence Measure if exist contradictions among the elements of ontology according to logical consistency [37] Computational Efficiency Measures the reasoner performance when processing the ontology [15] Conciseness About the existence of irrelevant information unnecessary and redundancies [15] Modularity Indicates if the ontology was created with an appropriate methodology in order to define if exist reusable components [8] Tangledness…”