Assessing the practice of biomedical ontology evaluation: Gaps and opportunities

Amith, Muhammad; He, Zhe; Bian, Jiang; Lossio-Ventura, Juan Antonio; Tao, Cui

doi:10.1016/j.jbi.2018.02.010

Cited by 76 publications

(80 citation statements)

References 72 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We have previously noted that many NCBO ontologies do not display documented evidence of evaluation, perhaps indicating lack of validating and verification of the underlying ontological knowledge [4]. This suggests that software tools for ontology evaluation are not readily assessable by the ontology research community, or when available, are not easy to use.…”

Section: Introductionmentioning

confidence: 99%

Architecture and usability of OntoKeeper, an ontology evaluation tool

Amith

Manion

Liang

et al. 2019

BMC Med Inform Decis Mak

Self Cite

View full text Add to dashboard Cite

Background The existing community-wide bodies of biomedical ontologies are known to contain quality and content problems. Past research has revealed various errors related to their semantics and logical structure. Automated tools may help to ease the ontology construction, maintenance, assessment and quality assurance processes. However, there are relatively few tools that exist that can provide this support to knowledge engineers. Method We introduce OntoKeeper as a web-based tool that can automate quality scoring for ontology developers. We enlisted 5 experienced ontologists to test the tool and then administered the System Usability Scale to measure their assessment. Results In this paper, we present usability results from 5 ontologists revealing high system usability of OntoKeeper, and use-cases that demonstrate its capabilities in previous published biomedical ontology research. Conclusion To the best of our knowledge, OntoKeeper is the first of a few ontology evaluation tools that can help provide ontology evaluation functionality for knowledge engineers with good usability.

show abstract

Section: Introductionmentioning

confidence: 99%

Architecture and usability of OntoKeeper, an ontology evaluation tool

Amith

Manion

Liang

et al. 2019

BMC Med Inform Decis Mak

Self Cite

View full text Add to dashboard Cite

show abstract

“…During phase 1, only 47% (n=88) of the concept matches were the same between the 2 coders; this increased to 100% in phase 2. Although the percentage agreement was low at the beginning, statistically suggesting weakness in the initial findings [50], the science of clinical informatics is still maturing and has yet to demonstrate how this value fully impacts the reliability of mapping results [51]. It is possible that this lower agreement was related to large number of target concepts (eg, ICHI beta-1 version had 7000 concepts), differences in concept understanding (eg, differences between counseling, advising, education, and emotional support), and different levels of experience in mapping ICNP and ICHI content.…”

Section: Discussionmentioning

confidence: 99%

Visibility of Community Nursing Within an Administrative Health Classification System: Evaluation of Content Coverage

Block¹,

Currie²,

Hardiker³

et al. 2019

J Med Internet Res

View full text Add to dashboard Cite

Background The World Health Organization is in the process of developing an international administrative classification for health called the International Classification of Health Interventions (ICHI). The purpose of ICHI is to provide a tool for supporting intervention reporting and analysis at a global level for policy development and beyond. Nurses represent the largest resource carrying out clinical interventions in any health system. With the shift in nursing care from hospital to community settings in many countries, it is important to ensure that community nursing interventions are present in any international health information system. Thus, an investigation into the extent to which community nursing interventions were covered in ICHI was needed. Objective The objectives of this study were to examine the extent to which International Classification for Nursing Practice (ICNP) community nursing interventions were represented in the ICHI administrative classification system, to identify themes related to gaps in coverage, and to support continued advancements in understanding the complexities of knowledge representation in standardized clinical terminologies and classifications. Methods This descriptive study used a content mapping approach in 2 phases in 2018. A total of 187 nursing intervention codes were extracted from the ICNP Community Nursing Catalogue and mapped to ICHI. In phase 1, 2 coders completed independent mapping activities. In phase 2, the 2 coders compared each list and discussed concept matches until consensus on ICNP-ICHI match and on mapping relationship was reached. Results The initial percentage agreement between the 2 coders was 47% (n=88), but reached 100% with consensus processes. After consensus was reached, 151 (81%) of the community nursing interventions resulted in an ICHI match. A total of 36 (19%) of community nursing interventions had no match to ICHI content. A total of 100 (53%) community nursing interventions resulted in a broader ICHI code, 9 (5%) resulted in a narrower ICHI code, and 42 (23%) were considered equivalent. ICNP concepts that were not represented in ICHI were thematically grouped into the categories family and caregivers, death and dying, and case management. Conclusions Overall, the content mapping yielded similar results to other content mapping studies in nursing. However, it also found areas of missing concept coverage, difficulties with interterminology mapping, and further need to develop mapping methods.

show abstract

“…There are many different ways to evaluate the quality of ontologies 60,61 . Our objective was to identify ontologies that might be re-usable for the domain of behaviour change and evaluate their content rather than their structure, corresponding to the task that Katsumi and Gruninger call "Search" 62 .…”

Section: Quality Assessment Of Identified Ontologiesmentioning

confidence: 99%

A scoping review of ontologies related to human behaviour change

et al. 2019

View full text Add to dashboard Cite

Ontologies are classification systems specifying entities, definitions and inter-relationships for a given domain, with the potential to advance knowledge about human behaviour change. A scoping review was conducted to: i) identify what ontologies exist related to human behaviour change, ii) describe the methods used to develop these ontologies and iii) assess the quality of identified ontologies. Using a systematic search, 2303 papers were identified. Fifteen ontologies met the eligibility criteria for inclusion, developed in areas such as cognition, mental disease and emotions. Methods used for developing the ontologies were expert consultation, data-driven techniques and re-use of terms from existing taxonomies, terminologies and ontologies. Best practices used in ontology development and maintenance were documented. The review did not identify any ontologies representing the breadth and detail of human behaviour change. This suggests that advancing behavioural science would benefit from the development of a behaviour change intervention ontology.

show abstract

Assessing the practice of biomedical ontology evaluation: Gaps and opportunities

Cited by 76 publications

References 72 publications

Architecture and usability of OntoKeeper, an ontology evaluation tool

Architecture and usability of OntoKeeper, an ontology evaluation tool

Visibility of Community Nursing Within an Administrative Health Classification System: Evaluation of Content Coverage

A scoping review of ontologies related to human behaviour change

Contact Info

Product

Resources

About