Model Checking Human–Human Communication Protocols Using Task Models and Miscommunication Generation

Abstract

Human-human communication is critical to safe operations in air transportation systems. For example, airlines develop and train pilots to use communication protocols designed to ensure that verbally communicated air traffic clearances are correctly executed by the pilots. Given the safety criticality of such interactions, these protocols should be designed to be robust to miscommunication. However, designers may not anticipate all of the different ways that miscommunication can occur. Thus, communication protocols can fail. This paper presents a method for evaluating human-human communication protocols using the enhanced operator function model with communications, which is a task analytic modeling formalism that can be used in model-checking formal verification analyses. In particular, a novel means of generating miscommunications from normative human-human communication protocols, instantiated as enhanced operator function models with communications, is introduced. An air transportation example is used to illustrate the power of the approach, where a protocol is iteratively evaluated and improved to make it robust for multiple miscommunications. Different versions of the application protocol are used to assess the scalability of the method. Results are discussed, and avenues of future research are explored. B. Formal Methods for Human-Human Communication Behavior Although human-human communication protocols could be modeled using traditional formal methods approaches, human-human communication, which can include verbal statements, gestures, and related actions, is different from machine communication. In this context, human communications are actions [18] that occur as part of the participants' larger tasks. For analysis and engineering purposes, tasks are usually documented as part of a task analysis [19] and are designed to capture the goal-directed normative behaviors humans use to accomplish goals with The work documented here is an extended version of results originally presented at the Fifth NASA Formal Methods Symposium in May 2013 at the NASA Ames Research Center titled "Evaluating Human-Human Communication Protocols with Miscommunication Generation and Model Checking." This new paper extends the work from original proceedings by iteratively applying the described miscommunication generation technique and the EOFMC-supported formal verification analyses to multiple versions of the original protocol to improve protocol robustness. It also reports scalability benchmarks and features an expanded introduction and discussion.

Task-based Automated Test Case Generation for Human-machine Interaction

“…Future work should investigate which coverage criteria provide analysts with the most useful information. Additionally, EOFM has a variant called EOFMC (EOFM with Communica- (Bass et al, 2011) that allows human-human communication and coordination to be incorporated into large task and formal system models both with and without miscommunication generation (Bolton, 2015). Future work should investigate how human-human communication and coordinated can be incorporated into our test generation method.…”

Modeling Operator Behavior in the Safety Analysis of Collaborative Robotic Applications

“…Currently, model-based techniques that include human errors focus on interface devices and not physical collaboration and contacts [10]. For example, Physiograms [19] model interfaces of physical devices and [9,36] study the impacts of miscommunication in human-human collaboration while interacting with critical systems. [7,38] explore human deviation from correct instructions using ConcurTaskTrees [37].…”

Novel Developments in Formal Methods for Human Factors Engineering

“…However, because they are extremely good at finding unexpected problems with interactions between components in complex environments, a growing body of research has been investigating how formal methods (and especially model checking) can be used in human factors engineering (Bolton, Bass, & Siminiceanu, 2013;Weyers, Bowen, Dix, & Palanque, 2017) to find problems in human-automation and human-human interaction. These works have predominantly focused on analyzing the usability of human-machine and human-computer interfaces (Abowd, Wang, & Monk, 1995;Campos & Harrison, 2008;Pa-ternò, 1997); finding potential mode confusions and automation surprises (Bredereke & Lankenau, 2002;Campos & Harrison, 2011;Degani, 2004;Degani & Heymann, 2002;Joshi, Miller, & Heimdahl, 2003;Rushby, 2002); assessing the impact of normative human task behavior on system safety (Aït-Ameur & Baron, 2006;Bolton, Siminiceanu, & Bass, 2011;Houser, Ma, Feigh, & Bolton, 2015;Paternò & Santoro, 2001); assessing the impact of human errors included (Bastide & Basnyat, 2007;Fields, 2001) or generated (Bolton, 2015;Bolton, Bass, & Siminiceanu, 2012;Pan & Bolton, 2016) in task analytic behavior models; or having problems arise organically from cognitive or perceptual models (Cerone, Lindsay, & Connelly, 2005;Hasanain, Boyd, & Bolton, 2015;Hasanain, Boyd, Edworthy, & Bolton, 2017;Rukšėnas, Curzon, Back, & Blandford, 2007).…”