Systematic and Realistic Testing in Simulation of Control Code for Robots in Collaborative Human-Robot Interactions

Abstract

The software of robotic assistants needs to be verified, to ensure its safety and functional correctness. Testing in simulation allows a high degree of realism in the verification. However, generating tests that cover both interesting foreseen and unforeseen scenarios in human-robot interaction (HRI) tasks, while executing most of the code, remains a challenge. We propose the use of belief-desire-intention (BDI) agents in the test environment, to increase the level of realism and human-like stimulation of simulated robots. Artificial intelligence, such as agent theory, can be exploited for more intelligent test generation. An automated testbench was implemented for a simulation in Robot Operating System (ROS) and Gazebo, of a cooperative table assembly task between a humanoid robot and a person. Requirements were verified for this task, and some unexpected design issues were discovered, leading to possible code improvements. Our results highlight the practicality of BDI agents to automatically generate valid and human-like tests to get high code coverage, compared to hand-written directed tests, pseudorandom generation, and other variants of model-based test generation. Also, BDI agents allow the coverage of combined behaviours of the HRI system with more ease than writing temporal logic properties for model checking.

A corroborative approach to verification and validation of human–robot teams

“…For this reason, we stimulate the robot's code in the simulation indirectly through stimulating its environment (e.g., the person's behaviour) instead, and we use a combination of model-based and pseudorandom test generation. Also, to alleviate the complexity of generating and timing different types of system inputs, the test generator is based on a two-tiered approach (Araiza-Illan et al, 2016) where an abstract test is generated first and then concretized by instantiating low-level parameters. The high-level actions of the human in the simulator include sending signals to the robot or setting abstract parameters for gaze, location and pressure.…”

Ros-gazebo. una valiosa Herramienta de Vanguardia para el Desarrollo de la Robótica

“…También es importante indicar que con el binomio ROS-Gazebo es posible realizar códigos de verificación del correcto funcionamiento del robot, mediante alertas de fallas detectadas [29]. Ejemplo de ello, es el código de comprobación de estabilización PID para un robot humanoide, utilizado en el entorno Gazebo [30].…”

Intelligent Agent-Based Stimulation for Testing Robotic Software in Human-Robot Interactions

“…Model-based approaches explore requirement or test models to achieve biasing automatically and systematically, e.g. with model checking guided by temporal logic properties representing realistic use cases [2,3]. Constructing models and exploring them automatically reduces the need to write constraints by hand.…”

“…Integrating comprehensive testing capabilities into popular robotics software development frameworks increases quality and compliance assurance at design time, and thus brings developers closer to achieve demonstrably safe robots. We implemented these testbenches in the Robot Operating System 1 (ROS) framework, and the Gazebo 2 3-D physics simulator, via the following components: a driver, self-checkers (assertion monitors executed in parallel with the robot's code), a coverage collector (based on code, assertion and crossproduct coverage models), and a test generator [2,3]. The test generation process makes use of pseudorandom, constrained, and model-based methods to produce abstract tests (sequences or programs), subsequently "concretized" by valid parameter instantiation.…”

“…Examples of the testbenches in ROS-Gazebo are available online. 3 Our previous model-based test generation techniques were based on model checking probabilistic timed automata (PTA) with respect to reachability temporal logic properties [2,3]. Although these have been very effective in guiding test generation to achieve high levels of coverage, both, the PTA models, often at very high abstraction levels, as well as suitable properties are required, which limits the approach in practice.…”