A corroborative approach to verification and validation of human–robot teams

Abstract

We present an approach for the verification and validation (V&V) of robot assistants in the context of human-robot interactions (HRI), to demonstrate their trustworthiness through corroborative evidence of their safety and functional correctness. Trust in robot assistants will allow them to transition from the laboratory into our everyday lives. Key challenges include the complex and unpredictable nature of the real world in which assistant and service robots operate, the limitations on available V&V techniques when used individually, and the consequent lack of confidence in the V&V results. Our approach, called corroborative V&V, addresses these challenges by combining several different V&V techniques; in this paper we use formal verification (model checking), simulation-based testing, and user validation in experiments with a real robot. We demonstrate our corroborative V&V approach through a handover task, the most critical part of a complex cooperative manufacturing scenario, for which we propose some safety and liveness requirements to verify and validate. We construct formal models, simulations and an experimental test rig for the HRI. To capture requirements we use temporal logic properties, assertion checkers and textual descriptions. This combination of approaches allows V&V of the HRI task at different levels of modelling detail and thoroughness of exploration, thus overcoming the individual limitations of each technique. Should the resulting V&V evidence present discrepancies, an iterative process between the different V&V techniques takes place until corroboration between the V&V techniques is gained from refining and improving the assets (i.e., system and requirement models) to represent the HRI task in a more truthful manner. Therefore, corroborative V&V affords a systematic approach to "'meta-V&V," in which different V&V techniques can be used to corroborate and check one another, increasing the level of certainty in the results of V&V.

Taxonomy of Trust-Relevant Failures and Mitigation Strategies

“…End user experiments can be used to test aspects such as trustworthiness. Formal verification, simulation based testing and end user experiments can help improve the safety, reliability and trust in robotic systems [23,77], as well as help mitigate as system failure (all), design failure and expectation failure (end user experiments).…”

“…Software engineering techniques such as verification and validation can be used to ensure that the system conforms to its requirements (verification) and the system meets the need of the stakeholder (validation). This improves reliability, safety and trustworthiness of the systems (see for example [23,77]) and will help mitigate some of the failures leading to loss of trust.…”

Robotics and Integrated Formal Methods: Necessity Meets Opportunity

“…The heterogeneous nature of robotic systems where various programming languages are used in the implementation of distinct components of the system means that this final specification-to-code step is not trivial. Since humans interact with these systems, it is also necessary to model the human component of the system, however, the verification of human behaviour is largely beyond the reach of current formal verification approaches [34,35].…”

“…We do not ignore the important role that validation techniques, such as testing and simulation, play in the development of robotic systems. These too, should be integrated into the development process to be used alongside formal methods [35].…”

“…Furthermore, several views of the same system or component often require integration of analysis, even for one specification element. For example, model-checking, model-based testing, and user evaluation have all been applied to the same robotic system [35]. Each, however, works at a very different level of abstraction and formality and so the challenge here is to integrate this breadth of techniques in a holistic framework.…”