This paper conducts a study on closed-loop control of engine performance parameters during mode transition process of TBCC engine based on artificial intelligence method. Firstly, a composite modeling method based on stepwise regression analysis and batch normalization-depth neural network is proposed to establish the on-board model during mode transition to estimate the thrust and inlet airflow in real-time. Secondly, based on the hybrid penalty function-particle swarm optimization algorithm, a mode transition control schedule applicable to the closed-loop control of thrust and inlet airflow is developed. Finally, a data processing method based on similarity conversion is proposed to extend the applicable envelope range of the mode transition control system. The transition time is shortened by 33.3 %, and the fluctuations of thrust and inlet airflow are reduced by 1.33 % and 10.77 %, respectively. When the control system is applied to the off-design mode transition process, a satisfactory mode transition performance is also obtained.