Unmanned Aircraft Systems Airspace Integration: A Game Theoretical Framework for Concept Evaluations

Abstract

The focus of this paper is to present a game theoretical modeling framework for the integration of unmanned aircraft systems into the National Airspace System. The problem of predicting the outcome of complex scenarios, where manned and unmanned air vehicles coexist, is the research problem of this work. The fundamental gap in the literature is that the models of interaction between manned and unmanned vehicles are insufficient: 1) They assume that pilot behavior is known a priori, and 2) They disregard pilot reaction and the decision-making process. The contribution of this paper is to propose a realistic modeling framework that will fill this gap. The foundations of the proposed method are formed by game theory, which investigates strategic decision making between intelligent agents; bounded rationality concept, which is based on the fact that humans cannot always make perfect decisions; and reinforcement learning, which is shown to be effective in human learning in psychology literature. An analysis of integration is conducted using an example scenario in the presence of manned aircraft and fully autonomous unmanned aircraft systems equipped with sense-and-avoid algorithms.Nomenclature m = message P = probability Q = value function for state-action pairs R = minimum safety distance, nm r = relative position vector between the unmanned aircraft system and the intruder, nm r m = minimum relative position vector between the unmanned aircraft system and the intruder, nm r 0 = initial relative position vector between the unmanned aircraft system and the intruder, nm s = state V = value function for states ϵ = learning rate ζ = angle between r and v AB , rad v = velocity vector, m∕s v A = velocity vector of the unmanned aircraft system, m∕s v B = velocity vector of the intruder, m∕s v AB = relative velocity vector between the unmanned aircraft system and the intruder, m∕s v x = X component of the velocity vector, m∕s v y = Y component of the velocity vector, m∕s v d A = velocity adjustment command vector for the unmanned aircraft system, m∕s π· = policy Ψ = heading angle, rad Ψ d = desired heading angle, rad

Modeling cyber-physical human systems via an interplay between reinforcement learning and game theory

“…This result was a continuation of leading studies conducted by Oyler et al (2016) and Li et al (2016). Similar to Musavi et al (2016) and Musavi et al (2018), this contribution is also the first in the automotive literature where a large number of decision making drivers are simultaneously modeled using a game theoretical modeling approach. Recently, an extended version of this work, which covers a larger class of interaction scenarios, with the help of a road traffic simulation on a 5-lane highway, is presented in Albaba et al (2019), where validation studies by processing real traffic data, which is provided in Colyar and Halkias (2007), is conducted.…”

“…However, although this approach presents some insight into the dynamics of the system, it is limiting due to grossly simplifying the real-life human interactions. The study conducted by Musavi et al (2016) provided, for the first time in the literature, probabilistic outcomes of UAS integration scenarios, where each of the 180 aircraft pilots are modeled using the game theoretical decision making process, simultaneously. In this research, Bayesian Networks are not utilized and the probabilistic decision making is obtained through employing a stochastic reinforcement learning algorithm proposed in Jaakkola et al (1994).…”

“…A hybrid airspace scenario where red squares represent manned aircraft and the cyan square is used to indicate an unmanned aircraft (UA). The yellow circles in the 600 km x 300 km airspace are the waypoints assigned to the UA (Musavi et al (2016),. reprinted with permission of the American Institute of Aeronautics and Astronautics, Inc.)…”

Game Theoretic Modeling of Driver and Vehicle Interactions for Verification and Validation of Autonomous Vehicle Control Systems

“…In [27] and [28], hierarchical reasoning decision-making was exploited to predict pilot behavior, and reasonable predictions were observed. In this paper, we employ this formalism to model driver interactions in traffic, and will show some validations of the developed driver models in Section V.…”

“…On the other hand, the hierarchical reasoning-based game theoretic approach exploited in this paper is easily scalable. Indeed, an implementation of the proposed framework for modeling human pilotto-unmanned aerial vehicle interactions with 50 players and with 180 players can be found in [27] and [28], and scenarios with up to 30 road vehicles are handled in this paper.…”