Online Monitoring for Safe Pedestrian-Vehicle Interactions

Du, Peter; Huang, Zhe; Liu, Tianqi; Ji, Tianchen; Xu, Ke; Gao, Qichao; Sibai, Hussein; Driggs-Campbell, Katherine; Mitra, Sayan

doi:10.1109/itsc45102.2020.9294366

Cited by 17 publications

(14 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recall our motivating example in Section 3.3, we study the Polaris GEM e2 Electric Vehicle and its high-fidelity Gazebo simulation [14].…”

Section: Case Study 1: Vision-based Lane Keeping With Lanenetmentioning

confidence: 99%

See 1 more Smart Citation

Verifying Controllers with Convolutional Neural Network-based Perception: A Case for Intelligible, Safe, and Precise Abstractions

Hsieh¹,

Joshi²,

Misailovíc³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Convolutional Neural Networks (CNN) for object detection, lane detection, and segmentation now sit at the head of most autonomy pipelines, and yet, their safety analysis remains an important challenge. Formal analysis of perception models is fundamentally difficult because their correctness is hard if not impossible to specify. We present a technique for inferring intelligible and safe abstractions for perception models from system-level safety requirements, data, and program analysis of the modules that are downstream from perception. The technique can help tradeoff safety, size, and precision, in creating abstractions and the subsequent verification. We apply the method to two significant case studies based on highfidelity simulations (a) a vision-based lane keeping controller for an autonomous vehicle and (b) a controller for an agricultural robot. We show how the generated abstractions can be composed with the downstream modules and then the resulting abstract system can be verified using program analysis tools like CBMC. Detailed evaluations of the impacts of size, safety requirements, and the environmental parameters (e.g., lighting, road surface, plant type) on the precision of the generated abstractions suggest that the approach can help guide the search for corner cases and safe operating envelops.

show abstract

“…Recall our motivating example in Section 3.3, we study the Polaris GEM e2 Electric Vehicle and its high-fidelity Gazebo simulation [14].…”

Section: Case Study 1: Vision-based Lane Keeping With Lanenetmentioning

confidence: 99%

“…To prepare the training data for learning A 𝑖 and b 𝑖 to construct R 𝑖 , we use the Gazebo simulator in [14] to generate camera images p labeled with their ground truth percepts z * . Each image is sampled from an uniform distribution D over the subspace X 𝑖 as well as the environment space 𝐸.…”

Section: Tracking Error Function Descriptionmentioning

confidence: 99%

Verifying Controllers with Convolutional Neural Network-based Perception: A Case for Intelligible, Safe, and Precise Abstractions

Hsieh¹,

Joshi²,

Misailovíc³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…A comprehensive understanding of dynamic human environments is essential for autonomous mobile robots to safely and smoothly enter our daily lives [Chen et al, 2017]. The mobile robot needs to effectively encode motion patterns of surrounding pedestrians from observation data, accurately predict their future trajectories, and efficiently plan its own paths for rapid task execution free of safety risks [Ziebart et al, 2009, Du et al, 2020. Social awareness is crucial for robots to achieve such human-centered autonomy, and significant progress has been made to acquire this capability by studying human-human interaction and predicting trajectories of multiple pedestrians [Alahi et al, 2016, Vemula et al, 2018, Gupta et al, 2018, Ivanovic and Pavone, 2019.…”

Section: Introductionmentioning

confidence: 99%

Learning Sparse Interaction Graphs of Partially Detected Pedestrians for Trajectory Prediction

Huang,

Li,

Shin

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Multi-pedestrian trajectory prediction is an indispensable safety element of autonomous systems that interact with crowds in unstructured environments. Many recent efforts have developed trajectory prediction algorithms with focus on understanding social norms behind pedestrian motions. Yet we observe these works usually hold two assumptions that prevent them from being smoothly applied to robot applications: positions of all pedestrians are consistently tracked; the target agent pays attention to all pedestrians in the scene. The first assumption leads to biased interaction modeling with incomplete pedestrian data, and the second assumption introduces unnecessary disturbances and leads to the freezing robot problem. Thus, we propose Gumbel Social Transformer, in which an Edge Gumbel Selector samples a sparse interaction graph of partially observed pedestrians at each time step. A Node Transformer Encoder and a Masked LSTM encode the pedestrian features with the sampled sparse graphs to predict trajectories. We demonstrate that our model overcomes the potential problems caused by the assumptions, and our approach outperforms the related works in benchmark evaluation.

show abstract

“…As mobile robots are becoming prevalent in people's daily lives, autonomous navigation in crowded places with other dynamic agents is an important yet challenging problem [1], [2]. Inspired by the recent applications of deep learning in robot control [3]- [6] and in graph modeling [7], we seek to build a learning-based graphical model for mobile robot navigation in pedestrian-rich environments.…”

Section: Introductionmentioning

confidence: 99%

“…We present the following contributions: (1) We propose a novel deep neural network architecture called DS-RNN, which enables the robot to perform efficient spatio-temporal reasoning in crowd navigation; (2) We train the network using model-free RL without any supervision, which both simplifies the learning pipeline and avoids the network from converging to a suboptimal policy too early; (3) Our method demonstrates better performance in challenging navigation settings compared with previous methods 1 .…”

Section: Introductionmentioning

confidence: 99%

Decentralized Structural-RNN for Robot Crowd Navigation with Deep Reinforcement Learning

Liu¹,

Chang²,

Weihang³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Safe and efficient navigation through human crowds is an essential capability for mobile robots. Previous work on robot crowd navigation assumes that the dynamics of all agents are known and well-defined. In addition, the performance of previous methods deteriorates in partially observable environments and environments with dense crowds. To tackle these problems, we propose decentralized structural-Recurrent Neural Network (DS-RNN), a novel network that reasons about spatial and temporal relationships for robot decision making in crowd navigation. We train our network with model-free deep reinforcement learning without any expert supervision. We demonstrate that our model outperforms previous methods and successfully transfer the policy learned in the simulator to a real-world TurtleBot 2i.

show abstract

Online Monitoring for Safe Pedestrian-Vehicle Interactions

Cited by 17 publications

References 46 publications

Verifying Controllers with Convolutional Neural Network-based Perception: A Case for Intelligible, Safe, and Precise Abstractions

Verifying Controllers with Convolutional Neural Network-based Perception: A Case for Intelligible, Safe, and Precise Abstractions

Learning Sparse Interaction Graphs of Partially Detected Pedestrians for Trajectory Prediction

Decentralized Structural-RNN for Robot Crowd Navigation with Deep Reinforcement Learning

Contact Info

Product

Resources

About