An Analysis of Displays for Probabilistic Robotic Mission Verification Results

Robotics and Autonomous Systems

et al. 2017

h i g h l i g h t s• This paper addresses the challenges involved in building a software tool for automatically verifying the behavior of multi-robot waypoint missions using formal methods.• Missions can include uncertainly located obstacles and uncertain environment geometry as well as uncertainty in robot motion.• We leverage a unique approach, VIPARS, to verifying performance guarantees for autonomous behavior-based robot software based on a combination of static analysis and Bayesian networks.• Two approaches to modeling probabilistic localization for verification are presented: a high-level approach and an approach that allows run-time localization code to be embedded within verification.• Verification and experimental validation results are presented for several autonomous robot missions, demonstrating the accuracy of verification and the mission-specific benefit of localization. a b s t r a c tEstablishing a-priori mission performance guarantees is crucial if autonomous robots are to be used with confidence in missions where failure could incur high costs in life and property damage. Automatic mission software verification, in addition to simulation and experimental benchmarking, is a key component of the solution for establishing performance guarantees. This component requires automatically verifying that the software constructed by the mission designer when executed in a partially known environment will adhere to the performance guarantee. In prior work we developed VIPARS, a unique approach to verifying performance guarantees for autonomous behavior-based robot software based on a combination of static analysis and Bayesian networks. While that approach produced fast and accurate verification of single robot missions with robot motion uncertainty, it did not address multiple-robot missions or any form of uncertainty related to environment geometry. This paper addresses the challenges involved in building a software tool for verifying the behavior of a multi-robot waypoint mission that includes uncertainly located obstacles and uncertain environment geometry as well as uncertainty in robot motion. An approach is presented to the problem of a-priori specification of uncertain environments for robot program verification. Two approaches to modeling probabilistic localization for verification are presented: a high-level approach and an approach that allows run-time localization code to be embedded in verification. Verification and experimental validation results are presented for several autonomous robot missions, demonstrating the accuracy of verification and the mission-specific benefit of localization

Section: Discussionmentioning

confidence: 99%

Performance verification for robot missions in uncertain environments

Robotics and Autonomous Systems

et al. 2017

“…The performance guarantee is quantified as a probability distribution that represents the robot mission's likelihood for success. These results also serve as the basis for performance feedback; and how this information should ultimately be presented to the mission operator was investigated in our recent human subjects study [23].…”

Section: B Validationmentioning

confidence: 99%

Formal Performance Guarantees for Behavior-Based Localization Missions

2016 IEEE 28th International Conference on Tools With Artificial Intelligence (ICTAI)

et al. 2016

Localization and mapping algorithms can allow a robot to navigate well in an unknown environment. However, whether such algorithms enhance any specific robot mission is currently a matter for empirical validation. In this paper we apply our MissionLab/VIPARS mission design and verification approach to an autonomous robot mission that uses probabilistic localization software.Two approaches to modeling probabilistic localization for verification are presented: a high-level approach, and a samplebased approach which allows run-time code to be embedded in verification. Verification and experimental validation results are presented for two waypoint missions using each method, demonstrating the accuracy of verification, and both are compared with verification of an odometry-only mission, to show the mission-specific benefit of localization.

“…A software module based on the approach, VIPARS, has been used to establish probabilistic performance guarantees for multiple-robot missions [3], for missions with probabilistic obstacle information [4] as well as missions using probabilistic localization software [5]. We have also studied the usability of our system [6].…”

Section: Introductionmentioning

confidence: 99%

Formal performance guarantees for an approach to human in the loop robot missions

2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC)

et al. 2017

A key challenge in the automatic verification of robot mission software, especially critical mission software, is to be able to effectively model the performance of a human operator and factor that into the formal performance guarantees for the mission. We present a novel approach to modelling the skill level of the operator and integrating it into automatic verification using a linear Gaussians model parameterized by experimental calibration. Our approach allows us to model different skill levels directly in terms of the behavior of the lumped, robot plus operator, system. Using MissionLab and VIPARS (a behavior-based robot mission verification module), we present a comparison of our predicted performance guarantees for two missions in which a teleoperated quadrotor identifies a target for an autonomous ground robot to intercept: one mission in which the operator flies the quadrotor by line of sight to locate the target and one where the operator flies the quadrotor using its video feed. We demonstrate the effectiveness of our approach by comparing predicated performance to experimentally measured performance.