The experimental evaluation of knowledge acquisition techniques and methods: history, problems and new directions

Abstract

The special problems of experimentally evaluating knowledge acquisition and knowledge engineering tools, techniques and methods are outlined, and illustrated in detail with reference to two series of studies. The "rst is a series of experiments undertaken at Nottingham University under the aegis of the UK Alvey initiative and the ESPRIT project ACKnowledge. The second is the series of Sisyphus benchmark studies. A suggested programme of experimental evaluation is outlined which is informed by the problems with using Sisyphus for evaluation.

Machine Biological Clock: Exploring the Time Dimension in an Organic-Based Operating System

“…It is well known that a significant obstacle to the construction of knowledge-based systems is the process of knowledge acquisition (Shadbolt, O'hara, & Crow, 1999). The key to this process is how we may effectively acquire the knowledge that will be implemented in the knowledge base.…”

AKTiveSA: A Technical Demonstrator System For Enhanced Situation Awareness

“…This was subsumed within a large-scale ontology engineering initiative that drew on state-of-the-art approaches to knowledge elicitation [15,16], knowledge modelling [17] and knowledge representation [18].…”

“…Knowledge capture assumed the form of direct observation of military planning activities and also entailed the use of interview techniques with subsequent protocol analysis of the interview transcripts. We also relied on a variety of knowledge acquisition techniques, such as laddered grids and process maps [15,16] to organize and structure knowledge gleaned from a variety of 2 Mine action activities and post-strike information in fact constitute a focal point for knowledge modelling in the DIF DTC Phase II SEMIOTIKS cluster project, a successor to the AKTiveSA initiative.…”

Knowledge society arguments revisited in the semantic technologies era

“…In the knowledge engineering domain, the issue of regulation has been tackled under the term knowledge certification in 1990s (Shadbolt et al, 1999) in the context of certified knowledge bases. Recently, the issue of using ontologies as a commodity, and their commercial interest has been debated (O'Hara and Shadbolt, 2001).…”