Human-machine conversations to support multi-agency missions

Abstract

In domains such as emergency response, environmental monitoring, policing and security, sensor and information networks are deployed to assist human users across multiple agencies to conduct missions at or near the "front line". These domains present challenging problems in terms of human-machine collaboration: human users need to task the network to help them achieve mission objectives, while humans (sometimes the same individuals) are also sources of mission-critical information. We propose a natural language-based conversational approach to supporting humanmachine working in mission-oriented sensor networks. We present a model for human-machine and machine-machine interactions in a realistic mission context, and evaluate the model using an existing surveillance mission scenario. The model supports the flow of conversations from full natural language to a form of Controlled Natural Language (CNL) amenable to machine processing and automated reasoning, including high-level information fusion tasks. We introduce a mechanism for presenting the gist of verbose CNL expressions in a more convenient form for human users. We show how the conversational interactions supported by the model include requests for expansions and explanations of machine-processed information.

Cognitive computing for coalition situational understanding

“…The CC approach that directly addresses the D attribute is MAS, allowing heterogeneous AI/SP components to be dynamically composed across a coalition network, as exemplified (for the D CH M HS problem case) in [17]. KRR also plays a role here, providing ontological commitments to support robust communication and interoperability in a heterogeneous system, D H .…”

“…This approach is exemplified in our previous [17] and current [23], [24] work, managing both the H CH attribute and also the hybrid nature of a CC system approach.…”

Agile development of ontologies through conversation

“…In earlier work [3] we developed a simple model of conversational interaction based on speech act theory and embodied through the creation of a number of "card" concepts with different specializations which enable conversations to flow between human and machine agents. The cards themselves are expressed as CE instances and they contain a "payload" which is the content of the conversational act as well as a rich set of meta-data including the date/time of the speech act, the card which it is in reply to, the user (human or machine) which created it and the user(s) (human or machine) to which it is sent.…”

“…In our previous work we have experimented with Controlled Natural Languages [1] , specifically ITA Controlled English (CE) [2] in an attempt to present ontological information in a more human-friendly, consumable form, aimed directly at non-technical business users. In our most recent work we have demonstrated a full natural language conversational capability [3] that enables human users to interact with a Controlled Natural Language Knowledge Base in their own natural language (e.g. English) in order to assert new knowledge, explore the current knowledge base (both in terms of the "model" and the "facts") and direct their questions to specific human or machine agents [3] .…”

“…In our most recent work we have demonstrated a full natural language conversational capability [3] that enables human users to interact with a Controlled Natural Language Knowledge Base in their own natural language (e.g. English) in order to assert new knowledge, explore the current knowledge base (both in terms of the "model" and the "facts") and direct their questions to specific human or machine agents [3] . Our initial experiments with human subjects demonstrate that untrained users are able to assert basic knowledge into the knowledge base and ask questions in both solitary (offline) and cooperative group (online) settings [4] .…”