Deep reinforcement learning achieves multifunctional morphing airfoil control

Abstract

Smooth camber morphing aircraft offer increased control authority and improved aerodynamic efficiency. Smart material actuators have become a popular driving force for shape changes, capable of adhering to weight and size constraints and allowing for simplicity in mechanical design. As a step towards creating uncrewed aerial vehicles (UAVs) capable of autonomously responding to flow conditions, this work examines a multifunctional morphing airfoil’s ability to follow commands in various flows. We integrated an airfoil with a morphing trailing edge consisting of an antagonistic pair of macro fiber composites (MFCs), serving as both skin and actuator, and internal piezoelectric flex sensors to form a closed loop composite system. Closed loop feedback control is necessary to accurately follow deflection commands due to the hysteretic behavior of MFCs. Here we used a deep reinforcement learning algorithm, Proximal Policy Optimization, to control the morphing airfoil. Two neural controllers were trained in a simulation developed through time series modeling on long short-term memory recurrent neural networks. The learned controllers were then tested on the composite wing using two state inference methods in still air and in a wind tunnel at various flow speeds. We compared the performance of our neural controllers to one using traditional position-derivative feedback control methods. Our experimental results validate that the autonomous neural controllers were faster and more accurate than traditional methods. This research shows that deep learning methods can overcome common obstacles for achieving sufficient modeling and control when implementing smart composite actuators in an autonomous aerospace environment.

Deep learning reduces sensor requirements for gust rejection on a small uncrewed aerial vehicle morphing wing

“…3e ). We found that the controller speed was not significantly affected by the pressure tap configurations ( P > 0.05) for all flight conditions and was consistent with rise times established in previous work where DRL controllers showed to be faster than traditional feedback control methods for an MFC morphing wing 42 (Fig. 3f–h ).…”

“…The two subsequent hidden layers were structured linearly with 512 nodes each and rectified linear unit (ReLU) activation functions 60 , 61 . Due to challenges and time constraints associated with DRL training in hardware environments, many hyperparameters were selected based on previous work performed in a similar MFC morphing environment 42 (Supplementary Table 2 ). However, we tuned the learning rate manually, determining a value of 3 × 10 −5 to be suitable for Adam optimization 62 .…”

“…Alternatively, model-free deep reinforcement learning (DRL) can train neural networks to make action decisions directly from raw sensor inputs without using dynamics or state inference models 40 , 41 . Proximal policy optimization (PPO) is a DRL algorithm that has shown to account for MFC hysteresis and produce effective camber control in a morphing airfoil 42 , 43 . For this reason, we used PPO to develop the GA policies (i.e., controllers) directly from three different sensor combinations (Supplementary Fig.…”

Nature, smart structures, and morphing UAVs

“…Maneuverability depends on the ability to quickly change velocity (both speed and direction). Other often neglected aspects of agility and maneuverability in translating avian inspired ideas to UAVs is the speed of actuation and the computing time required to quickly respond 5,6 .…”

Active deployment of ultra-thin composite booms with piezoelectric actuation

“…MFCs contain piezoceramic fibers embedded in epoxy matrix, which are sandwiched between two polyimide films with interdigitated electrodes, and primarily strains in the fiber direction under voltage input. Due to their flexibility and thinness, MFCs are popular actuators for inducing shape change in morphing aircraft structures, [23][24][25] but they are seldom implemented in deployable space structures. To the authors' knowledge, the only notable examples are in the post-deployed shape correction of composite hinges 26 and reflectors, 27 but initiating deployment with piezoelectric actuation has not been explored.…”