Predicting the outcomes of cyber-physical systems with multiple human interactions is a challenging problem. This article reviews a game theoretical approach to address this issue, where reinforcement learning is employed to predict the time-extended interaction dynamics. We explain that the most attractive feature of the method is proposing a computationally feasible approach to simultaneously model multiple humans as decision makers, instead of determining the decision dynamics of the intelligent agent of interest and forcing the others to obey certain kinematic and dynamic constraints imposed by the environment. We present two recent exploitations of the method to model 1) unmanned aircraft integration into the National Airspace System and 2) highway traffic. We conclude the article by providing ongoing and future work about employing, improving and validating the method. We also provide related open problems and research opportunities.

Recent advances in camera equipped drone applications and their widespread use increased the demand on vision based object detection algorithms for aerial images. Object detection process is inherently a challenging task as a generic computer vision problem, however, since the use of object detection algorithms on UAVs (or on drones) is relatively a new area, it remains as a more challenging problem to detect objects in aerial images. There are several reasons for that including: (i) the lack of large drone datasets including large object variance, (ii) the large orientation and scale variance in drone images when compared to the ground images, and (iii) the difference in texture and shape features between the ground and the aerial images. Deep learning based object detection algorithms can be classified under two main categories: (a) single-stage detectors and (b) multi-stage detectors. Both single-stage and multi-stage solutions have their advantages and disadvantages over each other. However, a technique to combine the good sides of each of those solutions could yield even a stronger solution than each of those solutions individually. In this paper, we propose an ensemble network, SyNet, that combines a multi-stage method with a single-stage one with the motivation of decreasing the high false negative rate of multi-stage detectors and increasing the quality of the single-stage detector proposals. As building blocks, CenterNet and Cascade R-CNN with pretrained feature extractors are utilized along with an ensembling strategy. We report the state of the art results obtained by our proposed solution on two different datasets: namely MS-COCO and visDrone with %52.1 mAPIoU=0.75 is obtained on MS-COCO val2017 dataset and %26.2 mAPIoU=0.75 is obtained on VisDrone test−set. Our code is available at: https://github.com/mertalbaba/SyNet

In this paper, a synergistic combination of deep reinforcement learning and hierarchical game theory is proposed as a modeling framework for behavioral predictions of drivers in highway driving scenarios. The need for a modeling framework that can address multiple human-human and human-automation interactions, where all the agents can be modeled as decision makers simultaneously, is the main motivation behind this work. Such a modeling framework may be utilized for the validation and verification of autonomous vehicles: It is estimated that for an autonomous vehicle to reach the same safety level of cars with drivers, millions of miles of driving tests are required. The modeling framework presented in this paper may be used in a high-fidelity traffic simulator consisting of multiple human decision makers to reduce the time and effort spent for testing by allowing safe and quick assessment of self-driving algorithms. To demonstrate the fidelity of the proposed modeling framework, game theoretical driver models are compared with real human driver behavior patterns extracted from traffic data.

This paper describes a stochastic modeling approach for predicting driver responses in highway traffic. Different from existing approaches in the literature, the proposed modeling framework allows simultaneous decision making for multiple drivers (>100), in a computationally feasible manner, instead of modeling the decisions of an ego driver and assuming a predetermined driving pattern for other drivers in a given scenario. This is achieved by a unique combination of hierarchical game theory, which is used to model strategic decision making, and stochastic reinforcement learning, which is employed to model multi-move decision making. The proposed approach can be utilized to create high fidelity traffic simulators, which can be used to facilitate the validation of autonomous driving control algorithms by providing a safe and relatively fast environment for initial assessment and tuning. What makes the proposed approach appealing especially for autonomous driving research is that the driver models are strategic, meaning that their responses are based on predicted actions of other intelligent agents in the traffic scenario, where these agents can be human drivers or autonomous vehicles. Therefore, these models can be used to create traffic models with multiple human-machine interactions. To evaluate the fidelity of the framework, created stochastic driver models are compared with real driving patterns, processed from the traffic data collected by US Federal Highway Administration on US101 (Hollywood Freeway) on June 15th, 2005.

No abstract