Powered parafoils, known for their impressive load-bearing capacity and extended endurance, have garnered significant interest. However, the parafoil system is a highly complex nonlinear system. It primarily relies on the steering gear to change flight direction and utilizes a thrust motor for climbing. However, achieving precise trajectory tracking control presents a challenge due to the interdependence of direction and altitude control. Furthermore, underactuation and wind disturbances bring additional difficulties for trajectory tracking control. Consequently, realizing trajectory tracking control for powered parafoils holds immense significance. In this paper, we propose a trajectory tracking method based on Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm-optimized Linear Active Disturbance Rejection Control (LADRC). Our method addresses the underactuation issue by incorporating a guiding law while utilizing two LADRC methods to achieve decoupling and compensate for disturbances. Moreover, we employ the TD3 algorithm to dynamically adjust controller parameters, thus enhancing the controller performance. The simulation results demonstrate the effectiveness of our proposed method as a trajectory tracking control approach. Additionally, since the control process is not reliant on system-specific models, our method can also provide guidance for trajectory tracking control in other aircraft.