BARK: Open Behavior Benchmarking in Multi-Agent Environments

Bernhard, Julian; Esterle, Klemens; Hart, Patrick; Keßler, Tobias

doi:10.1109/iros45743.2020.9341222

Cited by 33 publications

(20 citation statements)

References 23 publications

(35 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [24], Bernhard et al define the behavior of an autonomous vehicle as its desired future sequence of physical states encoding the agent's strategy to reach a shortterm goal, e.g changing lane. A behavior planner creates a behavior trajectory and passes it to a controller.…”

Section: B Interactive Planningmentioning

confidence: 99%

See 1 more Smart Citation

Risk-Constrained Interactive Safety under Behavior Uncertainty for Autonomous Driving

Bernhard¹,

Knoll²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Balancing safety and efficiency when planning in dense traffic is challenging. Interactive behavior planners incorporate prediction uncertainty and interactivity inherent to these traffic situations. Yet, their use of single-objective optimality impedes interpretability of the resulting safety goal. Safety envelopes which restrict the allowed planning region yield interpretable safety under the presence of behavior uncertainty, yet, they sacrifice efficiency in dense traffic due to conservative driving. Studies show that humans balance safety and efficiency in dense traffic by accepting a probabilistic risk of violating the safety envelope. In this work, we adopt this safety objective for interactive planning. Specifically, we formalize this safety objective, present the Risk-Constrained Robust Stochastic Bayesian Game modeling interactive decisions satisfying a maximum risk of violating a safety envelope under uncertainty of other traffic participants' behavior and solve it using our variant of Multi-Agent Monte Carlo Tree Search. We demonstrate in simulation that our approach outperforms baselines approaches, and by reaching the specified violation risk level over driven simulation time, provides an interpretable and tunable safety objective for interactive planning.

show abstract

Section: B Interactive Planningmentioning

confidence: 99%

“…We use the OpenSource behavior benchmarking environment BARK [24] for simulating two dense traffic scenarios:…”

Section: A Scenariosmentioning

confidence: 99%

Risk-Constrained Interactive Safety under Behavior Uncertainty for Autonomous Driving

Bernhard¹,

Knoll²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…We study our approach using the open-source benchmarking and development framework BARK proposed in [14]. We use Spot [15], a library for model checking, to translate the formalized LTL formula to a DFA, and to manipulate the automata.…”

Section: A Experimental Setupmentioning

confidence: 99%

Modeling and Testing Multi-Agent Traffic Rules within Interactive Behavior Planning

Esterle¹,

Gressenbuch²,

Knoll³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Autonomous vehicles need to abide by the same rules that humans follow. Some of these traffic rules may depend on multiple agents or time. Especially in situations with traffic participants that interact densely, the interactions with other agents need to be accounted for during planning. To study how multi-agent and time-dependent traffic rules shall be modeled, a framework is needed that restricts the behavior to ruleconformant actions during planning, and that can eventually evaluate the satisfaction of these rules. This work presents a method to model the conformance to traffic rules for interactive behavior planning and to test the ramifications of the traffic rule formulations on metrics such as collision, progress, or rule violations. The interactive behavior planning problem is formulated as a dynamic game and solved using Monte Carlo Tree Search, for which we contribute a new method to integrate history-dependent traffic rules into a decision tree. To study the effect of the rules, we treat it as a multi-objective problem and apply a relaxed lexicographical ordering to the vectorized rewards. We demonstrate our approach in a merging scenario. We evaluate the effect of modeling and combining traffic rules to the eventual compliance in simulation. We show that with our approach, interactive behavior planning while satisfying even complex traffic rules can be achieved. Moving forward, this gives us a generic framework to formalize traffic rules for autonomous vehicles.

show abstract

“…The proposed scenario is mixed up with junctions and roundabouts, while the traffic flow in the scenario is not adjustable. In [9], a multi-agent RL framework is proposed for the behavior model design under intersection scenarios, in which both rule-based and RL-based baselines are provided. The proposed simulator is similar to the one proposed in [10], both of them fail to provide a delicate dynamic model of vehicles and a highresolution simulator.…”

Section: Introductionmentioning

confidence: 99%

A Reinforcement Learning Benchmark for Autonomous Driving in Intersection Scenarios

Liu¹,

Zhang²,

Zhao³

2021

Preprint

View full text Add to dashboard Cite

In recent years, control under urban intersection scenarios becomes an emerging research topic. In such scenarios, the autonomous vehicle confronts complicated situations since it must deal with the interaction with social vehicles timely while obeying the traffic rules. Generally, the autonomous vehicle is supposed to avoid collisions while pursuing better efficiency. The existing work fails to provide a framework that emphasizes the integrity of the scenarios while being able to deploy and test reinforcement learning(RL) methods. Specifically, we propose a benchmark for training and testing RL-based autonomous driving agents in complex intersection scenarios, which is called RL-CIS. Then, a set of baselines are deployed consists of various algorithms. The test benchmark and baselines are to provide a fair and comprehensive training and testing platform for the study of RL for autonomous driving in the intersection scenario, advancing the progress of RL-based methods for intersection autonomous driving control. The code of our proposed framework can be found at https://github.com/liuyuqi123/ ComplexUrbanScenarios.

show abstract

BARK: Open Behavior Benchmarking in Multi-Agent Environments

Cited by 33 publications

References 23 publications

Risk-Constrained Interactive Safety under Behavior Uncertainty for Autonomous Driving

Risk-Constrained Interactive Safety under Behavior Uncertainty for Autonomous Driving

Modeling and Testing Multi-Agent Traffic Rules within Interactive Behavior Planning

A Reinforcement Learning Benchmark for Autonomous Driving in Intersection Scenarios

Contact Info

Product

Resources

About