A Design of Simulation Environment for Small Fixed-wing Aircraft

Abstract

Multi-Agent Particle Environment (MPE) [1] proposed by OpenAI is applied to the study of multi-agent reinforcement learning strategies. However, the motion rules of the agent are excessively simplified. In order to make the environment more suitable to small fixed-wing aircraft, we have made following improvements: 1. The dynamic model of the agent in the MPE does not conform to the characteristics of the fixed-wing aircraft. In order to simulate the dynamic characteristics of the fixed-wing aircraft, a speed-related damping mechanism is introduced into the two-dimensional motion environment. 2. Since the MPE lacks the control module for single agent, the MPE cannot meet the challenges raised by single agent control. A two-layer controller is proposed which includes the outer layer (Total Energy Control System and L_1) and the inner layer (PID). 3. The MPE does not contain any decision module. In order to comprehensively study the collaborative decision-making behavior of aircrafts in target allocation, a swarm decision module is added to the environment. In addition, the concept of control period is introduced to reduce the gap between simulation and the actual situation. Finally, several simulations were carried out to test the improved Multi-Agent Aircraft Environment (MAE). The test cases include the outer layer with L1 and Total Energy Control System (TECS) algorithm in two dimensions, the PID inner layer control algorithm and the designed auction algorithm. The tests complete the process of single aircraft flight, Multiple aircrafts scan-search flight and Multiple aircrafts dynamical-waypoint flight, which verifies the effectiveness of MAE.