Intention-Aware Autonomous Driving Decision-Making in an Uncontrolled Intersection

Abstract

Autonomous vehicles need to perform social accepted behaviors in complex urban scenarios including human-driven vehicles with uncertain intentions. This leads to many difficult decision-making problems, such as deciding a lane change maneuver and generating policies to pass through intersections. In this paper, we propose an intention-aware decision-making algorithm to solve this challenging problem in an uncontrolled intersection scenario. In order to consider uncertain intentions, we first develop a continuous hidden Markov model to predict both the high-level motion intention (e.g., turn right, turn left, and go straight) and the low level interaction intentions (e.g., yield status for related vehicles). Then a partially observable Markov decision process (POMDP) is built to model the general decision-making framework. Due to the difficulty in solving POMDP, we use proper assumptions and approximations to simplify this problem. A human-like policy generation mechanism is used to generate the possible candidates. Human-driven vehicles' future motion model is proposed to be applied in state transition process and the intention is updated during each prediction time step. The reward function, which considers the driving safety, traffic laws, time efficiency, and so forth, is designed to calculate the optimal policy. Finally, our method is evaluated in simulation with PreScan software and a driving simulator. The experiments show that our method could lead autonomous vehicle to pass through uncontrolled intersections safely and efficiently.

Belief state planning for autonomously navigating urban intersections

“…Uncertainty is represented by the transition model and the observation model. An important source of uncertainty in the context of autonomous driving is the behavior of the other drivers [8], [9]. From a state s ∈ S of the environment, the agent takes an action a ∈ A to maximize the expected accumulation of reward r(s, a) over time.…”

“…Two different scenarios are considered: right-turn and left-turn model of the problem. A partially observable Markov decision process (POMDP) is a standard model for sequential problems with stochastic state transitions and sensor uncertainty [6]- [8]. One of the challenges in this approach is in representing and modeling the problem in a way that allows the planning algorithm to be tractable [9].…”

Safe Reinforcement Learning on Autonomous Vehicles

“…Related work has looked at learning policies for intersection handling [26], [27], [28], [29] however these approaches are restricted to simulation and do not investigate the issue of preserving safety throughout the learning process under uncertainty. Using prediction as a safety constraint does not necessarily require additional learning.…”

Navigating Occluded Intersections with Autonomous Vehicles Using Deep Reinforcement Learning

“…Online planners based on partially observable Monte Carlo Planning (POMCP) have been shown to handle intersections [12], but rely on the existence of an accurate generative model. Offline learning tackles the intersection problem, often by using Markov Decision Processes (MDP) in the back-end [13], [14].…”