Automatic validation and failure diagnosis of human-device interfaces using task analytic models and model checking

IEEE Trans. Human-Mach. Syst.

Jimenez

Paassen

et al. 2014

Self Cite

Human-automation interaction (HAI) is often a contributor to failures in complex systems. This is frequently due to system interactions that were not anticipated by designers and analysts. Model checking is a method of formal verification analysis that automatically proves whether or not a formal system model adheres to desirable specification properties. Task analytic models can be included in formal system models to allow HAI to be evaluated with model checking. However, previous work in this area has required analysts to manually formulate the properties to check. Such a practice can be prone to analyst error and oversight which can result in unexpected dangerous HAI conditions not being discovered. To address this, this paper presents a method for automatically generating specification properties from task models that enables analysts to use formal verification to check for system HAI problems they may not have anticipated. This paper describes the design and implementation of the method. An example (a pilot performing a before landing checklist) is presented to illustrate its utility. Limitations of this approach and future research directions are discussed.Index Terms-Formal methods, human-automation interaction (HAI), model checking, system safety, task analysis.

Section: Discussionmentioning

confidence: 99%

“…EOFM's semantics (see Fig. 3) enable specification properties to assert qualities about the execution state of task models [48]. Thus, we can use computational concepts to automatically generate LTL specifications that assert all three of the above items and thus allow them to be checked formally.…”

Section: Specification Generationmentioning

confidence: 99%

Automatically Generating Specification Properties From Task Models for the Formal Verification of Human–Automation Interaction

IEEE Trans. Human-Mach. Syst.

Jimenez

Paassen

et al. 2014

Self Cite

“…The EOFMC [24] (an extension of the EOFM [26,30,31]) is used to model human-human communication protocols. The EOFMC was employed for several reasons.…”

Section: A Human-human Communication Protocol Modelingmentioning

confidence: 99%

Model Checking Human–Human Communication Protocols Using Task Models and Miscommunication Generation

Journal of Aerospace Information Systems

2015

Self Cite

Human-human communication is critical to safe operations in air transportation systems. For example, airlines develop and train pilots to use communication protocols designed to ensure that verbally communicated air traffic clearances are correctly executed by the pilots. Given the safety criticality of such interactions, these protocols should be designed to be robust to miscommunication. However, designers may not anticipate all of the different ways that miscommunication can occur. Thus, communication protocols can fail. This paper presents a method for evaluating human-human communication protocols using the enhanced operator function model with communications, which is a task analytic modeling formalism that can be used in model-checking formal verification analyses. In particular, a novel means of generating miscommunications from normative human-human communication protocols, instantiated as enhanced operator function models with communications, is introduced. An air transportation example is used to illustrate the power of the approach, where a protocol is iteratively evaluated and improved to make it robust for multiple miscommunications. Different versions of the application protocol are used to assess the scalability of the method. Results are discussed, and avenues of future research are explored. B. Formal Methods for Human-Human Communication Behavior Although human-human communication protocols could be modeled using traditional formal methods approaches, human-human communication, which can include verbal statements, gestures, and related actions, is different from machine communication. In this context, human communications are actions [18] that occur as part of the participants' larger tasks. For analysis and engineering purposes, tasks are usually documented as part of a task analysis [19] and are designed to capture the goal-directed normative behaviors humans use to accomplish goals with The work documented here is an extended version of results originally presented at the Fifth NASA Formal Methods Symposium in May 2013 at the NASA Ames Research Center titled "Evaluating Human-Human Communication Protocols with Miscommunication Generation and Model Checking." This new paper extends the work from original proceedings by iteratively applying the described miscommunication generation technique and the EOFMC-supported formal verification analyses to multiple versions of the original protocol to improve protocol robustness. It also reports scalability benchmarks and features an expanded introduction and discussion.

“…To illustrate how this method can be used to discover potential system problems, we present a PCA pump programming application, extended from [43], [44], and [66]. A PCA pump is a medical device that allows patients to control the delivery of pain medication based on a prescription programmed into it by a human practitioner.…”

Section: Applicationmentioning

confidence: 99%

Generating Erroneous Human Behavior From Strategic Knowledge in Task Models and Evaluating Its Impact on System Safety With Model Checking

IEEE Trans. Syst. Man Cybern, Syst.

Bass

2013

Self Cite

Human-automation interaction, including erroneous human behavior, is a factor in the failure of complex, safety-critical systems. This paper presents a method for automatically generating formal task analytic models encompassing both erroneous and normative human behavior from normative task models, where the misapplication of strategic knowledge is used to generate erroneous behavior. Resulting models can be automatically incorporated into larger formal system models so that safety properties can be formally verified with a model checker. This allows analysts to prove that a human-automation interactive system (as represented by the formal model) will or will not satisfy safety properties with both normative and generated erroneous human behavior. Benchmarks are reported that illustrate how this method scales. The method is then illustrated with a case study: the programming of a patientcontrolled analgesia pump. In this example, a problem resulting from a generated erroneous human behavior is discovered. The method is further employed to evaluate the effectiveness of different solutions to the discovered problem. The results and future research directions are discussed.Index Terms-Formal methods, human error, humanautomation interaction (HAI), model checking, system safety, task analysis.