基于对抗强化学习的组合动力发动机模态转换智能鲁棒控制方法

doi:10.7527/S1000-6893.2026.33361

Abstract

Abstract:

To address the strong nonlinearity， multi-actuator coupling， and stringent safety and robustness requirements of combined power engines during modal transition， an intelligent robust control method for the modal transition process is investigated. Focusing on the coordinated satisfaction of thrust tracking performance and safety constraints， an intelligent control framework based on deep reinforcement learning is established， in which an adversarial training mechanism is introduced to enhance robustness against observation disturbances and uncertainties. Based on a component-level engine simulation modal， multi-input multi-output control strategies and stage-wise training environments are designed for different modal transition processes， enabling adaptive policy learning and robustness improvement through adversarial reinforcement learning. In addition， to cope with engine parameter variations， a distributed control architecture based on multi-agent reinforcement learning is developed， and controller training is carried out under a centralized training and distributed execution scheme. Furthermore， real-time performance is validated on a hardware-in-the-loop platform， showing that the control cycle meets millisecond-level real-time requirements. Simulation results demonstrate that， under typical modal transition conditions， the proposed control method achieves steady-state thrust tracking errors within 1%， while exhibiting superior performance in thrust fluctuation amplitudes during modal transition compared with conventional control approaches. Under observation disturbances and parameter deviations， safety constraints are consistently satisfied without violation. The results indicate that the proposed intelligent robust control method effectively improves control accuracy， safety， and robustness during modal transition of combined power engines， providing a feasible solution for intelligent control of wide-speed-range combined power propulsion systems.

Key words: combined power engine, modal transition, intelligent robust control, reinforcement learning, adversarial training

CLC Number:

V236

Zizhao CHENG, Lijun LIU, Junjian LIN. Intelligent robust control method for modal transition in combined power engines based on adversarial reinforcement learning[J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(15): 633361.

Figures/Tables 46

Fig.1

Fig.2

Table 1

Fig.3

Fig.4

Fig.5

Fig.6

Table 2

Control quantity and actuator trend of Modal transition 1

控制量	符号	作动趋势
涡喷通道分流板角度	$S p T r b 1, S p T r b 2$	分流板角度增大
涡喷通道堵锥行程	$C o n e T r b 1, C o n e T r b 2$	堵锥行程增加
引射通道分流板角度	$S p e j c$	分流板角度减小
引射通道堵锥行程	$C o n e e j c$	堵锥行程减小
涡喷发动机燃油	$W f T r b 1, W f T r b 2$	减少供油
引射火箭燃油	$W f R o c$	增加供油

Table 2

Table 3

Control quantity and actuator trend of Modal transition 2

控制量	符号	作动趋势
引射火箭燃油量	$W f R o c$	停止供油
亚燃补燃燃油量	$W f R a m$	增加供油

Table 3

Table 4

Control quantity and actuator trend of Modal transition 3

控制量	符号	作动趋势
引射通道分流板角度	$S p e j c$	分流板角度增大
引射通道堵锥行程	$C o n e e j c$	堵锥行程增大
亚燃补燃燃油量	$W f R a m$	停止供油
超燃冲压燃油量	$W f S c r$	增加供油

Table 4

Fig.7

Fig.8

Fig.9

Fig.10

Fig.11

Table 5

Observed variables of controller under each modal transition stage

模态转换阶段	观测变量	符号
模态转换1	推力误差	$e t h r u s t$
	推力误差积分	$e s t h r u s t$
	总推力	$T h r u s t$
	推力误差微分	$e d e r i v a t i o n$
模态转换2	推力误差	$e t h r u s t$
	推力误差积分	$e s t h r u s t$
	总推力	$T h r u s t$
	推力误差微分	$e d e r i v a t i o n$
模态转换3	推力误差	$e t h r u s t$
	推力误差积分	$e s t h r u s t$
	总推力	$T h r u s t$
	推力误差微分	$e d e r i v a t i o n$

Table 5

Table 6

Control variables of controller under each modal transition stage

模态转化阶段	控制变量	符号
模态转换1	涡喷通道分流板角度	$S p T r b 1, S p T r b 2$
	引射通道分流板角度	$S p E j c$
	涡喷发动机燃油量	$W f T r b 1, W f T r b 2$
	引射火箭供油量	$W f R o c$
	涡喷通道尾喷管堵锥行程	$C o n e T r b 1, C o n e T r b 2$
	引射通道尾喷管堵锥行程	$C o n e E j c$
模态转换2	引射火箭供油量	$W f R o c$
模态转换2	亚燃补燃室供油量	$W f R a m$
模态转换3	引射通道分流板角度	$S p E j c$
	亚燃补燃室供油量	$W f R a m$
	超燃燃烧室供油量	$W f S c r$
	共用尾喷管堵锥行程	$C o n e E j c$

Table 6

Table 7

Table 8

Table 9

Table 10

Proportionality coefficient of reward function for thrust error in each modal transition stage

模态	$k 1$	$k 2$	$k 3$	$k 4$	$k 5$	$k 7$	$k 8$
涡喷-引射	-800	-400	-200	-50	-20	5	10
引射-亚燃	-200	-100	-50	-30	-20	1	5
亚燃-超燃	-200	-100	-50	-20	-5	1	2

Table 10

Table 11

Proportionality coefficient of reward function for engine safety in each modal transition

模态	参数	数值
涡喷-引射	$a 1$	10
	$b 1$	5
	$γ 11$	2
	$γ 12$	0.5
引射-亚燃	$a 2$	2
	$b 2$	5
	$γ 21$	1
	$γ 22$	1
亚燃-超燃	$a 3$	2
	$b 3$	4
	$γ 31$	1
	$γ 32$	0.5

Table 11

Table 12

Disturbance parameters and parameter setting ranges

参数	符号	拉偏系数	拉偏系数范围
涡喷压气机流量数组	$W c A r r a y$	$H P C W c B i a s$	$[0.95,1.05]$
涡喷燃烧室燃烧效率	$B u r n e r E f f i c i e n c y$	$B u r n e r B i a s$	$[0.95,1.05]$
引射火箭出口总压强	$P 3$	$P 3_B i a s s e t$	$[0.95,1.05]$
引射火箭出口总流量	$m 3$	$m 3_B i a s s e t$	$[0.95,1.05]$
亚燃燃烧室出口总压强	$P 5$	$P 5_B i a s s e t$	$[0.95,1.05]$
亚燃燃烧室出口总流量	$m 5$	$m 5_B i a s s e t$	$[0.95,1.05]$
超燃进气道出口总压强	$P 1$	$I s o l e t P B i a s$	$[0.95,1.05]$
超燃进气道出口总温	$T 1$	$I s o l e t T B i a s$	$[0.95,1.05]$

Table 12

Fig.12

Fig.13

Fig.14

Fig.15

Fig.16

Fig.17

Fig.18

Fig.19

Fig.20

Fig.21

Fig.22

Fig.23

Table 13

Fig.24

Fig.25

Table 14

Fig.26

Fig.27

Fig.28

Fig.29

Fig.30

Fig.31

Table 15

References 28

[1]	潘锐，赵群力. 国外高超声速飞机发展分析［J］. 航空科学技术， 2023， 34（11）： 2-7.
	PAN R， ZHAO Q L. Analysis on the development of hypersonic aircraft abroad［J］. Aeronautical Science & Technology， 2023， 34（11）： 2-7 （in Chinese）.
[2]	CSANK J， THOMAS S. A turbine based combined cycle engine inlet model and mode transition simulation based on HiTECC tool［C］∥ 48th AIAA/ASME/SAE/ASEE Joint Propulsion Conference & Exhibit. Reston： AIAA， 2012.
[3]	李茜，崔艳林. 2020年高超声速推进技术发展综述［J］. 航空动力， 2021（2）： 20-23.
	LI Q， CUI Y L. Development review of hypersonic propulsion technology in 2020［J］. Aerospace Power， 2021（2）： 20-23 （in Chinese）.
[4]	O’NEILL S. Supersonic travel seeks an encore［J］. Engineering， 2022， 10： 1-3.
[5]	李文杰. 从研发项目看美国高超声速飞机发展策略［J］. 战术导弹技术， 2023（5）： 59-63， 72.
	LI W J. Analysis of development strategy of US hypersonic aircraft through research projects［J］. Tactical Missile Technology， 2023（5）： 59-63， 72 （in Chinese）.
[6]	李文杰，林旭斌. NASA启动高超声速飞机研究［J］. 飞航导弹， 2021（5）： 9-12.
	LI W J， LIN X B. NASA launches hypersonic aircraft research［J］. Aerodynamic Missile Journal， 2021（5）： 9-12 （in Chinese）.
[7]	PENEV P， ZAGORSKI N. Aerospace systems “aerial launch” at the modern stage［J］. Aerospace Research in Bulgaria， 2022， 34： 80-94.
[8]	LEISHMAN J G. History of rockets & space flight［M］∥ Introduction to aerospace flight vehicles. Daytona Beach： Embry-Riddle Aeronautical University， 2022.
[9]	EET-China. British military and industry jointly announce concept of military hypersonic aircraft demonstrator［EB/OL］. ［2026-01-13］. .
[10]	News Tencent. Overview of the development of foreign hypersonic vehicle technology in 2023［EB/OL］. ［2026-01-13］. .
[11]	马婧雪. 含安全边界的TBCC组合发动机控制研究［D］. 哈尔滨：哈尔滨工业大学， 2018.
	MA J X. Control study for over-under TBCC engine with safety boundaries［D］. Harbin： Harbin Institute of Technology， 2018 （in Chinese）.
[12]	林威全，许航瑞，兰旭东. TBCC发动机的发展历程及关键技术分析［J］. 清华大学学报（自然科学版）， 2024， 64（9）： 1521-1535.
	LIN W Q， XU H R， LAN X D. Development and key technology analysis of turbine-based combined cycle engine［J］. Journal of Tsinghua University （Science and Technology）， 2024， 64（9）： 1521-1535 （in Chinese）.
[13]	韦宝禧，凌文辉，冮强，等. TRRE发动机关键技术分析及推进性能探索研究［J］. 推进技术， 2017， 38（2）： 298-305.
	WEI B X， LING W H， GANG Q， et al. Analysis of key technologies and propulsion performance research of TRRE engine［J］. Journal of Propulsion Technology， 2017， 38（2）： 298-305 （in Chinese）.
[14]	赵俊杰，徐惊雷. 基于CFD/CSD的TRRE排气系统气-热耦合分析［J］. 机械制造与自动化， 2022， 51（2）： 103-106.
	ZHAO J J， XU J L. Fluid-thermal coupling analysis based on CFD/CSD of TRRE exhaust system［J］. Machine Building & Automation， 2022， 51（2）： 103-106 （in Chinese）.
[15]	程锋. 组合动力飞行器宽速域气动/推进一体化耦合建模方法研究［D］. 西安：西北工业大学， 2019.
	CHENG F. Study on wide speed range aerodynamics/propulsion integrated coupled modeling of combined propulsion vehicle［D］. Xi’an： Northwestern Polytechnical University， 2019 （in Chinese）.
[16]	郭峰，桂丰，尤延铖，等. 一种涡轮基组合动力的整机低速风洞试验研究［J］. 推进技术， 2019， 40（11）： 2436-2443.
	GUO F， GUI F， YOU Y C， et al. Low-speed wind tunnel test of a turbine-based combined cycle propulsion system［J］. Journal of Propulsion Technology， 2019， 40（11）： 2436-2443 （in Chinese）.
[17]	尹泽勇，尤延铖，朱呈祥，等. 面向高超声速民机的多通道双涡轮引射冲压组合动力［J］. 航空学报， 2023， 44（2）： 627181.
	YIN Z Y， YOU Y C， ZHU C X， et al. Multi-channel dual-turbine ejector ramjet combined propulsion for hypersonic civil aircraft［J］. Acta Aeronautica et Astronautica Sinica， 2023， 44（2）： 627181 （in Chinese）.
[18]	李富霖，陈敏，唐海龙，等. 考虑环境指标的变循环发动机控制规律设计［J］. 航空学报， 2025， 46（16）： 231624.
	LI F L， CHEN M， TANG H L， et al. Control law design for variable cycle engine considering environmental indicators［J］. Acta Aeronautica et Astronautica Sinica， 2025， 46（16）： 231624 （in Chinese）.
[19]	聂聆聪，牟春晖，李帅衡. 基于强化学习的宽速域冲压发动机燃烧室一维压力分布控制方法研究［J］. 推进技术， 2025， 46（6）： 235-245.
	NIE L C， MU C H， LI S H. One dimensional pressure distribution reinforcement learning control for ramjet combustor with wide range velocity［J］. Journal of Propulsion Technology， 2025， 46（6）： 235-245 （in Chinese）.
[20]	卢则鹏，王建锋，李清，等. 有限功率下航空发动机伺服机构最优控制［J］. 海军航空大学学报， 2025， 40（5）： 745-751， 790.
	LU Z P， WANG J F， LI Q， et al. Optimal control of servo mechanism for aero-engine under limited power constraints［J］. Journal of Naval Aviation University， 2025， 40（5）： 745-751， 790 （in Chinese）.
[21]	张悦，杨刚，谭慧俊，等. 外并联型进气道模态转换过程中低速通道再起动特性［J］. 航空学报， 2026， 47（6）： 132842.
	ZHANG Y， YANG G， TAN H J， et al. Restart characteristics of low-speed duct during mode transition in an over-under inlet［J］. Acta Aeronautica et Astronautica Sinica， 2026， 47（6）： 132842 （in Chinese）.
[22]	GAO W B， ZHOU X， PAN M X， et al. Acceleration control strategy for aero-engines based on model-free deep reinforcement learning method［J］. Aerospace Science and Technology， 2022， 120： 107248.
[23]	QIAN R R， FENG Y M， JIANG M M， et al. Design and realization of intelligent aero-engine DDPG controller［J］. Journal of Physics： Conference Series， 2022， 2195（1）： 012056.
[24]	GAO W B， PAN M X， ZHOU W X， et al. Aero-engine modeling and control method with model-based deep reinforcement learning［J］. Aerospace， 2023， 10（3）： 209.
[25]	ZHU J M， TANG W， DONG J H， et al. A virtual reinforcement learning method for aero-engine intelligent control［C］∥ 2023 8th International Conference on Automation， Control and Robotics Engineering （CACRE）. Piscataway： IEEE Press， 2023： 138-143.
[26]	HU Q K， ZHAO Y P. Aero-engine acceleration control using deep reinforcement learning with phase-based reward function［J］. Proceedings of the Institution of Mechanical Engineers， Part G： Journal of Aerospace Engineering， 2022， 236（9）： 1878-1894.
[27]	张栋，唐硕. 超燃冲压发动机燃烧室准一维建模与分析［J］. 弹道学报， 2015， 27（1）： 85-91.
	ZHANG D， TANG S. Quasi one-dimensional modeling and analysis of scramjet combustor［J］. Journal of Ballistics， 2015， 27（1）： 85-91 （in Chinese）.
[28]	戴黄山. TBCC发动机安全强化学习智能控制方法研究［D］. 厦门：厦门大学， 2023.
	DAI H S. Safety-oriented reinforcement learning control of TBCC engines［D］. Xiamen： Xiamen University， 2023 （in Chinese）.

马赫数Ma	模态
0~2.5	涡喷模态
1.8~2.5	涡喷-引射模态转换
1.8~3.3	引射模态
2.8~3.2	引射-亚燃模态转换
2.8~4.7	亚燃模态
4.3~4.7	亚燃-超燃模态转换
4.3~6.0	超燃模态

参数	数值
Actor网络隐含层神经元数量	64，64，64
Critic网络隐含层神经元数量	（40，40，40），（40，40）
Actor网络学习率	0.000 1
Critic网络学习率	0.001
Soft更新率	0.001
经验回放池规模	10⁶
批量采样数量N	128
奖励函数折扣因子γ	0.99
OU噪声标准差	0.4
OU噪声标准差衰减率	0.001

参数	数值
Actor网络隐含层神经元数量	40，40，30
Critic网络隐含层神经元数量	（30，30，30），（40，30）
Actor网络学习率	0.000 1
Critic网络学习率	0.001
Soft更新率	0.001
经验回放池规模	10⁶
批量采样数量N	128
奖励函数折扣因子γ	0.99
OU噪声标准差	0.3
OU噪声标准差衰减率	0.001

参数	数值
Actor网络隐含层神经元数量	30，30，30
Critic网络隐含层神经元数量	（40，40，30），（30，30）
Actor网络学习率	0.000 1
Critic网络学习率	0.001
Soft更新率	0.001
经验回放池规模	10⁶
批量采样数量N	128
奖励函数折扣因子γ	0.99
OU噪声标准差	0.4
OU噪声标准差衰减率	0.001

模态	K_P	K_I	K_D
涡喷模态	1.930	6.15	0.438 00
引射模态	4.360	1.69	0.003 26
亚燃模态	0.822	1.12	0.139 00
超燃模态	0.324	5.44	0

Intelligent robust control method for modal transition in combined power engines based on adversarial reinforcement learning

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 46

References 28

Related Articles 15

Recommended Articles

Metrics

Comments

模态转换阶段	控制器类型	最大推力偏差/%	稳态误差/%	扰动下推力波动/%	是否满足安全限制
模态转换1	智能鲁棒控制器	3.1	0.8	±1.2	是
	智能控制器	5.2	1.4	±3.9	是
	模型预测控制器	4.8	1.0	±2.1	否
	PID控制器	8.6	2.9	±6.1	否
模态转换2	智能鲁棒控制器	4.2	0.9	±1.6	是
	智能控制器	4.3	1.8	±2.8	是
	模型预测控制器	5.1	1.2	±1.8	是
	PID控制器	10.4	3.2	±3.0	否
模态转换3	智能鲁棒控制器	4.8	1.0	±2.1	是
	智能控制器	7.1	1.9	±3.6	否
	模型预测控制器	8.3	1.1	±2.6	是
	PID控制器	12.7	4.1	±4.2	否

模态转换阶段	仿真类型	最大推力偏差/%	瞬态误差均值/%	误差波动幅值/%
模态转换1	单模态	3.1	0.7	±2.0
模态转换1	HIL	3.8	2.1	±2.1
模态转换2	单模态	4.2	0.9	±2.4
模态转换2	HIL	4.3	2.5	±2.9
模态转换3	单模态	4.8	1.0	±3.0
模态转换3	HIL	4.9	2.8	±3.8

[1]	Yuheng LIU, Li YANG, Qilong HUANG. Optimizing air and missile defense strategies with explainable hierarchical reinforcement learning [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(8): 332786-332786.
[2]	Hao ZHANG, Jianing LIU, Zhi XU, Yuanxin YANG. Trajectory prediction method of incoming missiles based on improved inverse reinforcement learning in aircraft active defense mode [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(8): 332753-332753.
[3]	Wei XIONG, Dong ZHANG, Shuheng YANG, Zhi REN, Wenyi LIU. Manned/unmanned aerial vehicle collaborative interpretable method for intelligent air combat [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(7): 332547-332547.
[4]	Yanlong HAN, An ZHANG, Wenhao BI, Qiucen FAN, Tianle HOU. Intelligent decision-making of airborne terminal infrared composite jamming based on DACTM-PPO [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(7): 332759-332759.
[5]	Sihua GAO, Bingyang ZHAO, Jianfu LI. UAV complete data collection trajectory planning algorithm based on time window constraints [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(6): 332451-332451.
[6]	Yunxiao LIAN, Ni LI, Feng XIE, Pan ZHOU, Changyin DONG. A multi-UAV cooperative air combat decision-making method based on spatial-temporal information fusion [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(6): 332633-332633.
[7]	Lei ZHANG, Can TIAN, Fangqing WEN, Qinghe ZHANG, Han LIU. Multi-objective evolution with deep deterministic strategy gradient algorithm for mobile edge networks [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(3): 631880-631880.
[8]	Shuo HE, Jialin LIU, Ao SHEN, Saisai ZHU, Yuanyuan JIN, Lulu LI, Yafei LI, Mingliang XU. Multi-stage collaborative decision-making approach for dynamic scheduling of carrier-based aircraft support operations [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(15): 333077-333077.
[9]	Tichao XU, Wenyue MENG, Jian ZHANG. Trajectory planning of solar powered unmanned aerial vehicles based on multi-objective reinforcement learning [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(12): 332817-332817.
[10]	Wenlin LIU, Xikun HU, Ping ZHONG. Reinforcement learning-driven object detection method for degraded remote sensing images [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(10): 532861-532861.
[11]	Zan MA, Jie BAI, Liqin YAN, Yong CHEN, Shuguang SUN. Safety assessment for airborne intelligent avoidance system based on Bayesian optimization [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(1): 331973-331973.
[12]	Tao ZHANG, Pan LI, Zixu WANG, Zhenhua ZHU. Design of reward functions for helicopter attitude control in reinforcement learning [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(S1): 732184-732184.
[13]	Kaifang WAN, Zhilin WU, Yunhui WU, Haozhi QIANG, Yibo WU, Bo LI. Cooperative location of multiple UAVs with deep reinforcement learning in GPS-denied environment [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(8): 331024-331024.
[14]	Lingfeng JIANG, Xinkai LI, Hai ZHANG, Hanwei LI, Hongli ZHANG. Mapless navigation of UAVs in dynamic environments based on an improved TD3 algorithm [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(8): 331035-331035.
[15]	Min YANG, Guanjun LIU, Ziyuan ZHOU. Control of lunar landers based on secure reinforcement learning [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(3): 630553-630553.