考虑视场角约束的突防/打击一体化智能决策算法

doi:10.7527/S1000-6893.2026.33053

Abstract

Abstract:

Aiming at the integrated penetration/strike design problem of the terminal guidance phase of hypersonic reentry glide missiles considering constraints such as the detection field of view angle， a Lagrange-proximal policy optimization intelligent penetration decision algorithm based on the Constrained Markov Decision Process （CMDP）and an adaptive training method for multiple scenarios are proposed. Suppose the terminal guidance phase of the gliding missile adopts the biased proportional guidance law to strike the target. Taking the relative motion state between the interceptor and the gliding missile， and between the gliding missile and the target as the state space， and the rate of change of the biased acceleration as the action space， the reward function is designed by comprehensively considering the penetration/strike result of the gliding missile， the control energy consumption， the constraint satisfaction situation， and the velocity vector lead angle of the interceptor. The constraint cost function about the field of view angle is constructed， and the constrained Markov decision process of the penetration/strike problem is established. The constraints are introduced into the loss function of the policy network through the Lagrange multiplier， and the constraint cost Critic network is introduced to construct the penetration network. The Proximal Policy Optimization （PPO） algorithm is used to train the network to obtain the biased acceleration. Establish rules for classifying the complexity of combat scenarios， and propose an adaptive sampling training method for combat scenarios of “progressive learning in the early stage and more learning on difficult points in the later stage” to enhance the convergence speed of the penetration strategy and its generalization to different combat scenarios. Simulation results show that this intelligent penetration/strike integrated strategy can enable gliding missile to successfully penetrate and strike the target at the specified impact angle while meeting the field of view angle constraints throughout the process， and it has good generalization.

Key words: hypersonic glide missile, integrated penetration and strike, field of view angle constraint, constrained Markov decision process, scene adaptive sampling

CLC Number:

V249.1

Gaopeng TAN, Xiaofang WANG, Hai LIN. Integrated intelligent decision-making algorithm for penetration and strike considering field of view angle constraint[J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(15): 333053.

Figures/Tables 35

Fig.1

Fig.2

Fig.3

Table 1

Table 2

Fig.4

Table 3

Initial parameters of scene

对象	参数	初始值
目标	位置/km	（0， 0， 0）
滑翔弹	位置 $x 0$ /km	［-10， -1］
	位置 $y 0$ /km	［30，33］
	位置 $z 0$ /km	［-78， -72］
	速度 $V M 0$ /（m·s^-1）	［1 980， 2 000］
	弹道倾角 $θ M 0$ /（°）	［-3.0， 3.0］
	弹道偏角 $ψ V M 0$ /（°）	［-89， -84］
拦截弹	位置 $(x D 0, y D 0, z D 0)$ /km	（11.12， 0.02， -5.22）
拦截弹	发射速度 $V D 0$ /（m·s^-1）	48

Table 3

Table 4

Table 5

Hyperparameters for training penetration algorithm

参数	数值
折扣因子 $γ$	0.99
Actor学习率/10^-3	0.3
Critic学习率	0.001
Constraint Critic学习率	0.001
拉格朗日乘子 $λ$ 初始值	5
拉格朗日乘子 $λ$ 更新率	0.015
优化器	Adam
平滑因子 $ξ$	0.95
裁剪因子 $ε$	0.2

Table 5

Table 6

Interference acting on glide vehicle

干扰	大小
$d 1 / (r a d ⋅ s - 1)$	$0.1 N (0,1)$
$d 2 / (r a d ⋅ s - 1)$	$0.1 N (0,1)$

Table 6

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

Fig.11

Fig.12

Fig.13

Fig.14

Fig.15

Fig.16

Fig.17

Fig.18

Fig.19

Fig.20

Fig.21

Fig.22

Fig.23

Fig.24

Fig.25

Fig.26

Fig.27

Table 7

Performance comparison of penetration strategies with different training methods

训练方式	脱靶量/m			累计加速度平方/（m²·s^-3）			视场角约束违反时间/s
训练方式	平均值	标准差	最大值	平均值	标准差	最大值	平均值	标准差	最大值	出现时段
PL+DM	4.14	2.26	9.06	140 350	18 547	183 948	0	0.04	0.38	$[T 1 + 6.02, T 2 - 0.11]$
PL	4.24	2.63	9.23	143 089	18 899	185 156	0.02	0.17	1.62	$[T 1 + 5.93, T 2 + 0.26]$
NS	4.83	2.65	9.26	147 701	21 884	197 382	0.06	0.24	1.81	$[T 1 + 5.81, T 2 + 1.08]$

Table 7

Fig.28

References 29

[1]	张远龙，谢愈. 滑翔飞行器弹道规划与制导方法综述［J］. 航空学报， 2020， 41（1）： 023377.
	ZHANG Y L， XIE Y. Review of trajectory planning and guidance methods for gliding vehicles［J］. Acta Aeronautica et Astronautica Sinica， 2020， 41（1）： 023377 （in Chinese）.
[2]	DING Y B， YUE X K， CHEN G S， et al. Review of control and guidance technology on hypersonic vehicle［J］. Chinese Journal of Aeronautics， 2022， 35（7）： 1-18.
[3]	刘双喜，刘世俊，李勇，等. 国外高超声速飞行器及防御体系发展现状［J］. 空天防御， 2023， 6（3）： 39-51.
	LIU S X， LIU S J， LI Y， et al. Current developments in foreign hypersonic vehicles and defense systems［J］. Air & Space Defense， 2023， 6（3）： 39-51 （in Chinese）.
[4]	何磊，闫晓东，唐硕. 螺旋俯冲机动突防的制导律设计［J］. 航空学报， 2019， 40（5）： 322457.
	HE L， YAN X D， TANG S. Guidance law design for spiral-diving maneuver penetration［J］. Acta Aeronautica et Astronautica Sinica， 2019， 40（5）： 322457 （in Chinese）.
[5]	ZARCHAN P. Proportional navigation and weaving targets［J］. Journal of Guidance， Control， and Dynamics， 1995， 18（5）： 969-974.
[6]	IMADO F， MIWA S. Missile guidance algorithm against high-g barrel roll maneuvers［J］. Journal of Guidance， Control， and Dynamics， 1994， 17（1）： 123-128.
[7]	SUN A， LIU H. Multi-pursuer evasion［C］∥AIAA Guidance， Navigation and Control Conference and Exhibit. Reston： AIAA， 2008.
[8]	邵会兵. 滑翔飞行器滑翔能力智能预示与多约束制导研究［D］. 哈尔滨：哈尔滨工业大学， 2020： 75-83.
	SHAO H B. Gliding manoeuvrability intelligent prediction and multi-constrained guidance for gliding vehicles［D］. Harbin： Harbin Institute of Technology， 2020： 75-83 （in Chinese）.
[9]	王雨琪，宁国栋，王晓峰，等. 基于微分对策的临近空间飞行器机动突防策略［J］. 航空学报， 2020， 41（S2）： 724276.
	WANG Y Q， NING G D， WANG X F， et al. Maneuver penetration strategy of near space vehicle based on differential game［J］. Acta Aeronautica et Astronautica Sinica， 2020， 41（S2）： 724276 （in Chinese）.
[10]	YAN T， CAI Y L， XU B. Evasion guidance algorithms for air-breathing hypersonic vehicles in three-player pursuit-evasion games［J］. Chinese Journal of Aeronautics， 2020， 33（12）： 3423-3436.
[11]	陈中原，韦文书，陈万春. 基于强化学习的多发导弹协同攻击智能制导律［J］. 兵工学报， 2021， 42（8）： 1638-1647.
	CHEN Z Y， WEI W S， CHEN W C. Reinforcement learning-based intelligent guidance law for cooperative attack of multiple missiles［J］. Acta Armamentarii， 2021， 42（8）： 1638-1647 （in Chinese）.
[12]	WANG N Y， WANG X G， CUI N G， et al. Deep reinforcement learning-based impact time control guidance law with constraints on the field-of-view［J］. Aerospace Science and Technology， 2022， 128： 107765.
[13]	GRANDO R B， DE JESUS J C， KICH V A， et al. Double critic deep reinforcement learning for mapless 3D navigation of unmanned aerial vehicles［J］. Journal of Intelligent & Robotic Systems， 2022， 104（2）： 29.
[14]	LI X， WANG X G， ZHOU H Y. Entry guidance for spatial no-fly zones avoidance via model-based reinforcement learning［J］. Aerospace Science and Technology， 2024， 153： 109405.
[15]	SUN B， VAN KAMPEN E J. Reinforcement-learning-based adaptive optimal flight control with output feedback and input constraints［J］. Journal of Guidance， Control， and Dynamics， 2021， 44（9）： 1685-1691.
[16]	路坤锋，贾晨辉，黄旭，等. 面向变构型飞行器的强化学习位置姿态一体化控制方法［J］. 宇航学报， 2024， 45（7）： 1100-1110.
	LU K F， JIA C H， HUANG X， et al. Reinforcement learning-based integrated position and attitude control method towards morphing flight vehicles［J］. Journal of Astronautics， 2024， 45（7）： 1100-1110 （in Chinese）.
[17]	何湘远，尘军，郭昊，等. 基于深度强化学习的高速飞行器攻防博弈［J］. 航天控制， 2022， 40（4）： 76-83.
	HE X Y， CHEN J， GUO H， et al. Attack-defense game based on deep reinforcement learning for high speed vehicle［J］. Aerospace Control， 2022， 40（4）： 76-83 （in Chinese）.
[18]	JIANG L， NAN Y， ZHANG Y， et al. Anti-interception guidance for hypersonic glide vehicle： A deep reinforcement learning approach［J］. Aerospace， 2022， 9（8）： 424.
[19]	GUO Y H， JIANG Z J， HUANG H Q， et al. Intelligent maneuver strategy for a hypersonic pursuit-evasion game based on deep reinforcement learning［J］. Aerospace， 2023， 10（9）： 783.
[20]	LI X， WANG X G， ZHOU H Y， et al. A novel evasion guidance for hypersonic morphing vehicle via intelligent maneuver strategy［J］. Chinese Journal of Aeronautics， 2024， 37（5）： 441-461.
[21]	ZHAO W S， QUERALTA J P， WESTERLUND T. Sim-to-real transfer in deep reinforcement learning for robotics： A survey［C］∥2020 IEEE Symposium Series on Computational Intelligence （SSCI）. Piscataway： IEEE Press， 2021.
[22]	倪浩，章胜，刘福炜，等. 基于域随机化增强EfficientZero的无人机空战智能决策［J］. 控制与决策， 2025， 40（11）： 3273-3286.
	NI H， ZHANG S， LIU F W， et al. UAV air combat intelligent decision-making based on domain randomization enhanced EfficientZero［J］. Control and Decision， 2025， 40（11）： 3273-3286 （in Chinese）.
[23]	裴培，何绍溟，王江，等. 一种深度强化学习制导控制一体化算法［J］. 宇航学报， 2021， 42（10）： 1293-1304.
	PEI P， HE S M， WANG J， et al. Integrated guidance and control for missile using deep reinforcement learning［J］. Journal of Astronautics， 2021， 42（10）： 1293-1304 （in Chinese）.
[24]	WANG J W， ZHANG R. Terminal guidance for a hypersonic vehicle with impact time control［J］. Journal of Guidance， Control， and Dynamics， 2018， 41（8）： 1790-1798.
[25]	孙勇，段广仁，张卯瑞，等. 高超声速飞行器再入过程改进气动系数模型［J］. 系统工程与电子技术， 2011， 33（1）： 134-137.
	SUN Y， DUAN G R， ZHANG M R， et al. Modified aerodynamic coefficient models of hypersonic vehicle in reentry phase［J］. Systems Engineering and Electronics， 2011， 33（1）： 134-137 （in Chinese）.
[26]	高峰，唐胜景，师娇，等. 一种基于落角约束的偏置比例导引律［J］. 北京理工大学学报， 2014， 34（3）： 277-282.
	GAO F， TANG S J， SHI J， et al. A bias proportional navigation guidance law based on terminal impact angle constraint［J］. Transactions of Beijing Institute of Technology， 2014， 34（3）： 277-282 （in Chinese）.
[27]	ALTMAN E. Constrained Markov decision processes： Stochastic modeling［D］. New York： Routledge， 1999.
[28]	SCHULMAN J， WOLSKI F， DHARIWAL P， et al. Proximal policy optimization algorithms［DB/OL］. arXiv： 1707.06347； 2017.
[29]	法尔汉·A·法拉奇. 微分博弈论在导弹和自主系统制导中的应用［M］. 北京：北京理工大学出版社， 2018： 121-146.
	FARHAN A F. Differential game theory with applications to missiles and autonomous systems guidance［M］. Beijing： Beijing Institute of Technology Press， 2018： 121-146 （in Chinese）.

场景	是否被拦截	拦截弹脱靶量/m	是否命中目标	是否全程满足视场角约束
Case 1	是	［0，5］		否
Case 2	是	［0，5］		是
Case 3	是	［5，10］		否
Case 4	是	［5，10］		是
Case 5	否		否	否
Case 6	否		否	是
Case 7	否		是	否
Case 8	否		是	是

场景	复杂程度
Case 6	简单
Case 7	简单
Case 8	简单
Case 3	中等
Case 4	中等
Case 5	中等
Case 1	复杂
Case 2	复杂

网络	相关层	节点数	激活函数	正则化
Actor网络	输入层	8
	隐藏层1	256	ReLU	LN
	隐藏层2	256	ReLU	LN
	输出层	2	Tanh
Constraint Critic/ Critic网络	输入层	8
	隐藏层1	256	ReLU	BN
	隐藏层2	256	ReLU	Dropout（0.3）
	输出层	1	Tanh

Integrated intelligent decision-making algorithm for penetration and strike considering field of view angle constraint

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 35

References 29

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	. Geometric control allocation for tailless aircraft based on nonlinear attainable mo-ment set [J]. Acta Aeronautica et Astronautica Sinica, 0, (): 1-0.
[2]	. Cooperative Swarm Control Based on Physics-Aware Attention MADDPG [J]. Acta Aeronautica et Astronautica Sinica, 0, (): 1-0.
[3]	Xiayang Zhang Pei LI Shao-ie LV. Methods and Effectiveness Evaluation of Tightly-Coupled Dynamic Obstacle Avoidance and Flight Control for Helicopters [J]. Acta Aeronautica et Astronautica Sinica, 0, (): 1-0.
[4]	Tichao XU, Wenyue MENG, Jian ZHANG. Trajectory planning of solar powered unmanned aerial vehicles based on multi-objective reinforcement learning [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(12): 332817-332817.
[5]	Qi CHEN, Chang ZHANG, Hongjin ZHANG, Jiyu HU, Like ZHAO. Trajectory tracking control of multiple parafoil formation based on fuzzy PID [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(S1): 732897-732897.
[6]	Bin YANG, Jun BIAN, Jianchun ZHANG, Kexin GUO, Yi YANG. Distributed safety cooperative control for multiple UAVs under multi-source risks [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(S1): 733010-733010.
[7]	Yuhan ZHANG, Cheng XU, Hua JI, Mingjun WEI. Adaptive formation rendezvous control for aerial refueling based on total energy method [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(S1): 732997-732997.
[8]	Xiaolong HE, Qifeng CHEN, Wenkang HAO. Distributed control of UAVs formation based on passivity-based sliding mode method [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(S1): 732965-732965.
[9]	Dapeng ZHOU, Chong ZHEN, Xiaolei QU, Fei LUO. Hybrid active-passive fault-tolerant control for carrier landing subject to control surface effectiveness loss [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(11): 332544-332544.
[10]	Fei LUO, Biao JIANG, Yining GAO, Zhiyong HU, Hailiang LIU, Junhong ZHANG, Zikang SU. Design and wind tunnel test of an advanced controller for aerial refueling docking [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(11): 332793-332793.
[11]	. http://hkxb.buaa.edu.cn hkxb@buaa.edu.cn [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(11): 332810-332810.
[12]	Wenyu ZHOU, Yingjie WANG, Chenhao OUYANG, Zixuan ZHENG, Xiaokui YUE. Bearing-based collision-avoidance formation control of quadrotor UAVs under internal and external disturbances [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(9): 533224-533224.
[13]	. http://hkxb.buaa.edu.cn hkxb@buaa.edu.cn [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(9): 533305-533305.
[14]	Jie WANG, Ming YANG, Bailing TIAN, Bohao LI, Zhishi CHEN. Dynamic event-triggered prescribed performance formation control for hypersonic morphing vehicles [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(9): 532499-532499.
[15]	Yuan LU, Ke ZHANG, Bin JIANG. Hierarchical fault-tolerant formation control for unmanned helicopters based on fully-actuated system approach [J]. Acta Aeronautica et Astronautica Sinica, 2026, 47(9): 532279-532279.