Safety Assurances for Human-Robot Interaction via Confidence-aware Game-theoretic Human Models

Tian, Ran; Sun, Liting; Bajcsy, Andrea; Tomizuka, Masayoshi; Dragan, Anca D.

doi:10.1109/icra46639.2022.9812048

Cited by 23 publications

(6 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Alternatively, recent works have turned to online estimation of human intent to adapt the reachable set to correspond to the estimated human intent model [22]- [24], and therefore improve the soundness of the reachable set. However, these methods primarily focus on estimating parameters describing a human behavior prediction model to help reduce the overconservatism in an AV's planning algorithm.…”

Section: Related Workmentioning

confidence: 99%

Refining Obstacle Perception Safety Zones via Maneuver-Based Decomposition

Topan¹,

Chen²,

Szczerbicki³

et al. 2023

2023 IEEE Intelligent Vehicles Symposium (IV)

View full text Add to dashboard Cite

A critical task for developing safe autonomous driving stacks is to determine whether an obstacle is safetycritical, i.e., poses an imminent threat to the autonomous vehicle. Our previous work showed that Hamilton Jacobi reachability theory can be applied to compute interactiondynamics-aware perception safety zones that better inform an ego vehicle's perception module which obstacles are considered safety-critical. For completeness, these zones are typically larger than absolutely necessary, forcing the perception module to pay attention to a larger collection of objects for the sake of conservatism. As an improvement, we propose a maneuverbased decomposition of our safety zones that leverages information about the ego maneuver to reduce the zone volume. In particular, we propose a "temporal convolution" operation that produces safety zones for specific ego maneuvers, thus limiting the ego's behavior to reduce the size of the safety zones. We show with numerical experiments that maneuver-based zones are significantly smaller (up to 76% size reduction) than the baseline while maintaining completeness.

show abstract

Section: Related Workmentioning

confidence: 99%

Refining Obstacle Perception Safety Zones via Maneuver-Based Decomposition

Topan¹,

Chen²,

Szczerbicki³

et al. 2023

2023 IEEE Intelligent Vehicles Symposium (IV)

View full text Add to dashboard Cite

show abstract

“…A common approach is to compute "inevitable" collision sets (ICS), e.g., via Hamilton-Jacobi reachability computation [19], with selected assumptions on other agents' behaviors, and perform shielding where the AV will flag a situation as unsafe whenever it is close to entering the ICS and execute an appropriate evasive action (e.g., [1,20,21]). A primary challenge is selecting reasonable behavior assumptions for ICS computation to balance tractability, interpretability, and compatibility with real-world driving interactions [22,23]. Most of these approaches assume that other agents act in an adversarial manner leading to overly conservative ICS.…”

Section: Related Workmentioning

confidence: 99%

Task-Relevant Failure Detection for Trajectory Predictors in Autonomous Vehicles

Farid¹,

Veer²,

Ivanovic³

et al. 2022

Preprint

View full text Add to dashboard Cite

In modern autonomy stacks, prediction modules are paramount to planning motions in the presence of other mobile agents. However, failures in prediction modules can mislead the downstream planner into making unsafe decisions. Indeed, the high uncertainty inherent to the task of trajectory forecasting ensures that such mispredictions occur frequently. Motivated by the need to improve safety of autonomous vehicles without compromising on their performance, we develop a probabilistic run-time monitor that detects when a "harmful" prediction failure occurs, i.e., a task-relevant failure detector. We achieve this by propagating trajectory prediction errors to the planning cost to reason about their impact on the AV. Furthermore, our detector comes equipped with performance measures on the false-positive and the false-negative rate and allows for data-free calibration. In our experiments we compared our detector with various others and found that our detector has the highest area under the receiver operator characteristic curve.

show abstract

“…In human-robot interaction scenarios, the robot typically maintains a model of human behavior in order to aid in the prediction of their future states. In previous works, the robot modeled the human as a utility-maximizing agent with respect to learned reward functions for given state-action pairs [1], [22], [9]. However, these models typically use discrete action sets rather than considering a continuous distribution over actions.…”

Section: B Human Prediction Modelmentioning

confidence: 99%

Online Update of Safety Assurances Using Confidence-Based Predictions

Nakamura¹,

Bansal²

2022

Preprint

View full text Add to dashboard Cite

Robots such as autonomous vehicles and assistive manipulators are increasingly operating in dynamic environments and close physical proximity to people. In such scenarios, the robot can leverage a human motion predictor to predict their future states and plan safe and efficient trajectories. However, no model is ever perfect -when the observed human behavior deviates from the model predictions, the robot might plan unsafe maneuvers. Recent works have explored maintaining a confidence parameter in the human model to overcome this challenge, wherein the predicted human actions are tempered online based on the likelihood of the observed human action under the prediction model. This has opened up a new research challenge, i.e., how to compute the future human states online as the confidence parameter changes? In this work, we propose a Hamilton-Jacobi (HJ) reachability-based approach to overcome this challenge. Treating the confidence parameter as a virtual state in the system, we compute a parameter-conditioned forward reachable tube (FRT) that provides the future human states as a function of the confidence parameter. Online, as the confidence parameter changes, we can simply query the corresponding FRT, and use it to update the robot plan. Computing parameter-conditioned FRT corresponds to an (offline) high-dimensional reachability problem, which we solve by leveraging recent advances in data-driven reachability analysis. Overall, our framework enables online maintenance and updates of safety assurances in human-robot interaction scenarios, even when the human prediction model is incorrect. We demonstrate our approach in several safetycritical autonomous driving scenarios, involving a state-of-theart deep learning-based prediction model.

show abstract

Safety Assurances for Human-Robot Interaction via Confidence-aware Game-theoretic Human Models

Cited by 23 publications

References 29 publications

Refining Obstacle Perception Safety Zones via Maneuver-Based Decomposition

Refining Obstacle Perception Safety Zones via Maneuver-Based Decomposition

Task-Relevant Failure Detection for Trajectory Predictors in Autonomous Vehicles

Online Update of Safety Assurances Using Confidence-Based Predictions

Contact Info

Product

Resources

About