Exploiting Users’ Feedbacks - Towards a Task-based Evaluation of Application Ontologies Throughout Their Lifecycle

Pittet, Perrine; Barthélémy, Jérôme

doi:10.5220/0005608202630268

Cited by 10 publications

(9 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We have used three different ontology evolution approaches [ 50 ] to evaluate our automated ontology generation framework. First, we develop and experimentally apply our automated biomedical ontology generation algorithms to evaluate our framework based on Task-based Evaluation [ 51 , 52 ] using CDR corpus [ 53 ] and SemMedDB [ 54 ]. Second, we have done baseline ontology-based evaluation using Alzheimer’s disease ontology [ 55 ] as gold standard.…”

Section: Methodsmentioning

confidence: 99%

Linked open data-based framework for automatic biomedical ontology generation

2018

View full text Add to dashboard Cite

BackgroundFulfilling the vision of Semantic Web requires an accurate data model for organizing knowledge and sharing common understanding of the domain. Fitting this description, ontologies are the cornerstones of Semantic Web and can be used to solve many problems of clinical information and biomedical engineering, such as word sense disambiguation, semantic similarity, question answering, ontology alignment, etc. Manual construction of ontology is labor intensive and requires domain experts and ontology engineers. To downsize the labor-intensive nature of ontology generation and minimize the need for domain experts, we present a novel automated ontology generation framework, Linked Open Data approach for Automatic Biomedical Ontology Generation (LOD-ABOG), which is empowered by Linked Open Data (LOD). LOD-ABOG performs concept extraction using knowledge base mainly UMLS and LOD, along with Natural Language Processing (NLP) operations; and applies relation extraction using LOD, Breadth first Search (BSF) graph method, and Freepal repository patterns.ResultsOur evaluation shows improved results in most of the tasks of ontology generation compared to those obtained by existing frameworks. We evaluated the performance of individual tasks (modules) of proposed framework using CDR and SemMedDB datasets. For concept extraction, evaluation shows an average F-measure of 58.12% for CDR corpus and 81.68% for SemMedDB; F-measure of 65.26% and 77.44% for biomedical taxonomic relation extraction using datasets of CDR and SemMedDB, respectively; and F-measure of 52.78% and 58.12% for biomedical non-taxonomic relation extraction using CDR corpus and SemMedDB, respectively. Additionally, the comparison with manually constructed baseline Alzheimer ontology shows F-measure of 72.48% in terms of concepts detection, 76.27% in relation extraction, and 83.28% in property extraction. Also, we compared our proposed framework with ontology-learning framework called “OntoGain” which shows that LOD-ABOG performs 14.76% better in terms of relation extraction.ConclusionThis paper has presented LOD-ABOG framework which shows that current LOD sources and technologies are a promising solution to automate the process of biomedical ontology generation and extract relations to a greater extent. In addition, unlike existing frameworks which require domain experts in ontology development process, the proposed approach requires involvement of them only for improvement purpose at the end of ontology life cycle.

show abstract

Section: Methodsmentioning

confidence: 99%

Linked open data-based framework for automatic biomedical ontology generation

2018

View full text Add to dashboard Cite

show abstract

“…The evaluation by Yu et al [49] involves users in a browsing task based on the ontology in Wikipedia, which has been enriched in a different way. Another version of task-based evaluation is presented by Pittet et al [50], where users are able to enrich the ontology.…”

Section: Ontology Evaluationmentioning

confidence: 99%

Evaluating QualiCO: an ontology to facilitate qualitative methods sharing to support open science

Hocker

Bipat

McDonald

et al. 2021

J Internet Serv Appl

View full text Add to dashboard Cite

Qualitative science methods have largely been omitted from discussions of open science. Platforms focused on qualitative science that support open science data and method sharing are rare. Sharing and exchanging coding schemas has great potential for supporting traceability in qualitative research as well as for facilitating the reuse of coding schemas. In this study, we present and evaluate QualiCO, an ontology to describe qualitative coding schemas. Twenty qualitative researchers used QualiCO to complete two coding tasks. In our findings, we present task performance and interview data that focus participants’ attention on the ontology. Participants used QualiCO to complete the coding tasks, decreasing time on task, while improving accuracy, signifying that QualiCO enabled the reuse of qualitative coding schemas. Our discussion elaborates some issues that participants had and highlights how conceptual and prior practice frames their interpretation of how QualiCO can be used.

show abstract

“…The evaluation by Yu et al [45] involves users in a browsing task based on the ontology in Wikipedia, which has been enriched in a different way. Another version of task-based evaluation is presented by Pittet et al [46], where users are able to enrich the ontology,…”

Section: Ontology Evaluationmentioning

confidence: 99%

Evaluating QualiCO: An Ontology to Facilitate Qualitative Methods Sharing to Support Open Science

Hocker

Bipat

McDonald

et al. 2021

Preprint

View full text Add to dashboard Cite

Qualitative science methods have largely been omitted from discussions of open science. Platforms focused on qualitative science that support open science data and method sharing are rare. Sharing and exchanging coding schemas has great potential for supporting traceability in qualitative research as well as for facilitating the re-use of coding schemas. In this study, we describe and evaluate QualiCO, an ontology for qualitative coding schemas. QualiCO is designed to describe a wide range of qualitative coding schemas. Twenty qualitative researchers used QualiCO to complete two coding tasks. In our findings, we present task performance and interview data that focus participants’ attention on the ontology. Participants used QualiCO to complete the coding tasks, decreasing time on task, while improving accuracy, signifying that QualiCO enabled the reuse of qualitative coding schemas. Our discussion elaborates some issues that participants had and highlights how conceptual and prior practice frames their interpretation of how QualiCo can be used.

show abstract

Exploiting Users’ Feedbacks - Towards a Task-based Evaluation of Application Ontologies Throughout Their Lifecycle

Cited by 10 publications

References 14 publications

Linked open data-based framework for automatic biomedical ontology generation

Linked open data-based framework for automatic biomedical ontology generation

Evaluating QualiCO: an ontology to facilitate qualitative methods sharing to support open science

Evaluating QualiCO: An Ontology to Facilitate Qualitative Methods Sharing to Support Open Science

Contact Info

Product

Resources

About