A Game Theoretical Modeling and Simulation Framework For The Integration Of Unmanned Aircraft Systems In To The National Airspace

Abstract

The focus of this paper is to present a game theoretical modeling and simulation framework for the integration of Unmanned Aircraft Systems (UAS) into the National Airspace system (NAS). The problem of predicting the outcome of complex scenarios, where UAS and manned air vehicles co-exist, is the research problem of this work. The fundamental gap in the literature in terms of developing models for UAS integration into NAS is that the models of interaction between manned and unmanned vehicles are insufficient. These models are insufficient because a) they assume that human behavior is known a priori and b) they disregard human reaction and decision making process. The contribution of this paper is proposing a realistic modeling and simulation framework that will fill this gap in the literature. The foundations of the proposed modeling method is formed by game theory, which analyzes strategic decision making between intelligent agents, bounded rationality concept, which is based on the fact that humans cannot always make perfect decisions, and reinforcement learning, which is shown to be effective in human behavior in psychology literature. These concepts are used to develop a simulator which can be used to obtain the outcomes of scenarios consisting of UAS, manned vehicles, automation and their interactions. An analysis of the UAS integration is done with a specifically designed scenario for this paper. In the scenario, a UAS equipped with sense and avoid algorithm, moves along a predefined trajectory in a crowded airspace. Then the effect of various system parameters on the safety and performance of the overall system is investigated.

A 3D Game Theoretical Framework for the Evaluation of Unmanned Aircraft Systems Airspace Integration Concepts

“…In prior works [14], [15], authors have created HAS models with human decision making models, inspired by a game theoretical methodology known as semi-network-form games [12], where the pilot behavior was not assumed to be known a priori but obtained using 1) the level-k reasoning concept which is a game theoretical approach used to model multiple strategic player interactions, where it is assumed that humans have various levels of reasoning, level-0 being the lowest level, and 2) reinforcement learning, which helps model time extended decisions as opposed to assuming one-shot decision making. Although these studies introduced one of the very first examples of HAS models where several decision makers can be modeled simultaneously in a time-extended manner, they had two limitations: First, HAS models were developed for a 2-dimensional (2D) airspace.…”

“…In the proposed framework, these limitations are removed and a 3D HAS model is introduced where the strategic decision makers can modify their policies during interactions between each other. Therefore, compared to [14], [15] a much larger class of interactions can be modeled.…”

Driver Modeling Through Deep Reinforcement Learning and Behavioral Game Theory

“…For instance, a uniformly random selection of actions can be defined as level-0 policy [54]. In earlier studies, where approaches similar to the one proposed in this paper, level-0 policies are set as a persisting single action regardless of the state being observed [55], [56], [57], or as a conditional logic based on experience [58]. The level-0 policy used in this study is defined as 1) hard decelerate if the car in front is close and approaching; 2) decelerate if the car in front is close and stable or nominal and approaching; 3) accelerate if the car in front is nominal and movingaway or f ar and 4) maintain otherwise.…”

Act to Reason: A Dynamic Game Theoretical Model of Driving

“…Several approaches can be considered for designing a Level-0 agent, considering the fact that the decision-maker is non-strategic, and the decision making logic can be stochastic. The policy can be a uniform random selection mechanism (Shapiro et al, 2014), or can take a single action regardless of the observation (Musavi et al, 2017(Musavi et al, , 2016Yildiz et al, 2014), or it can be a conditional logic based on experience (Backhaus et al, 2013). Our level-0 policy approach is stochastic for the merging action, and rule-based for the rest of the actions.…”