ACTA AERONAUTICAET ASTRONAUTICA SINICA >
Integrated intelligent decision-making algorithm for penetration and strike considering field of view angle constraint
Received date: 2025-11-05
Revised date: 2025-11-24
Accepted date: 2026-01-13
Online published: 2026-01-21
Supported by
National Level Project
Aiming at the integrated penetration/strike design problem of the terminal guidance phase of hypersonic reentry glide missiles considering constraints such as the detection field of view angle， a Lagrange-proximal policy optimization intelligent penetration decision algorithm based on the Constrained Markov Decision Process （CMDP）and an adaptive training method for multiple scenarios are proposed. Suppose the terminal guidance phase of the gliding missile adopts the biased proportional guidance law to strike the target. Taking the relative motion state between the interceptor and the gliding missile， and between the gliding missile and the target as the state space， and the rate of change of the biased acceleration as the action space， the reward function is designed by comprehensively considering the penetration/strike result of the gliding missile， the control energy consumption， the constraint satisfaction situation， and the velocity vector lead angle of the interceptor. The constraint cost function about the field of view angle is constructed， and the constrained Markov decision process of the penetration/strike problem is established. The constraints are introduced into the loss function of the policy network through the Lagrange multiplier， and the constraint cost Critic network is introduced to construct the penetration network. The Proximal Policy Optimization （PPO） algorithm is used to train the network to obtain the biased acceleration. Establish rules for classifying the complexity of combat scenarios， and propose an adaptive sampling training method for combat scenarios of “progressive learning in the early stage and more learning on difficult points in the later stage” to enhance the convergence speed of the penetration strategy and its generalization to different combat scenarios. Simulation results show that this intelligent penetration/strike integrated strategy can enable gliding missile to successfully penetrate and strike the target at the specified impact angle while meeting the field of view angle constraints throughout the process， and it has good generalization.
Gaopeng TAN , Xiaofang WANG , Hai LIN . Integrated intelligent decision-making algorithm for penetration and strike considering field of view angle constraint[J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2026 , 47(15) : 333053 -333053 . DOI: 10.7527/S1000-6893.2026.33053
| [1] | 张远龙， 谢愈. 滑翔飞行器弹道规划与制导方法综述［J］. 航空学报， 2020， 41（1）： 023377. |
| ZHANG Y L， XIE Y. Review of trajectory planning and guidance methods for gliding vehicles［J］. Acta Aeronautica et Astronautica Sinica， 2020， 41（1）： 023377 （in Chinese）. | |
| [2] | DING Y B， YUE X K， CHEN G S， et al. Review of control and guidance technology on hypersonic vehicle［J］. Chinese Journal of Aeronautics， 2022， 35（7）： 1-18. |
| [3] | 刘双喜， 刘世俊， 李勇， 等. 国外高超声速飞行器及防御体系发展现状［J］. 空天防御， 2023， 6（3）： 39-51. |
| LIU S X， LIU S J， LI Y， et al. Current developments in foreign hypersonic vehicles and defense systems［J］. Air & Space Defense， 2023， 6（3）： 39-51 （in Chinese）. | |
| [4] | 何磊， 闫晓东， 唐硕. 螺旋俯冲机动突防的制导律设计［J］. 航空学报， 2019， 40（5）： 322457. |
| HE L， YAN X D， TANG S. Guidance law design for spiral-diving maneuver penetration［J］. Acta Aeronautica et Astronautica Sinica， 2019， 40（5）： 322457 （in Chinese）. | |
| [5] | ZARCHAN P. Proportional navigation and weaving targets［J］. Journal of Guidance， Control， and Dynamics， 1995， 18（5）： 969-974. |
| [6] | IMADO F， MIWA S. Missile guidance algorithm against high-g barrel roll maneuvers［J］. Journal of Guidance， Control， and Dynamics， 1994， 17（1）： 123-128. |
| [7] | SUN A， LIU H. Multi-pursuer evasion［C］∥AIAA Guidance， Navigation and Control Conference and Exhibit. Reston： AIAA， 2008. |
| [8] | 邵会兵. 滑翔飞行器滑翔能力智能预示与多约束制导研究［D］. 哈尔滨： 哈尔滨工业大学， 2020： 75-83. |
| SHAO H B. Gliding manoeuvrability intelligent prediction and multi-constrained guidance for gliding vehicles［D］. Harbin： Harbin Institute of Technology， 2020： 75-83 （in Chinese）. | |
| [9] | 王雨琪， 宁国栋， 王晓峰， 等. 基于微分对策的临近空间飞行器机动突防策略［J］. 航空学报， 2020， 41（S2）： 724276. |
| WANG Y Q， NING G D， WANG X F， et al. Maneuver penetration strategy of near space vehicle based on differential game［J］. Acta Aeronautica et Astronautica Sinica， 2020， 41（S2）： 724276 （in Chinese）. | |
| [10] | YAN T， CAI Y L， XU B. Evasion guidance algorithms for air-breathing hypersonic vehicles in three-player pursuit-evasion games［J］. Chinese Journal of Aeronautics， 2020， 33（12）： 3423-3436. |
| [11] | 陈中原， 韦文书， 陈万春. 基于强化学习的多发导弹协同攻击智能制导律［J］. 兵工学报， 2021， 42（8）： 1638-1647. |
| CHEN Z Y， WEI W S， CHEN W C. Reinforcement learning-based intelligent guidance law for cooperative attack of multiple missiles［J］. Acta Armamentarii， 2021， 42（8）： 1638-1647 （in Chinese）. | |
| [12] | WANG N Y， WANG X G， CUI N G， et al. Deep reinforcement learning-based impact time control guidance law with constraints on the field-of-view［J］. Aerospace Science and Technology， 2022， 128： 107765. |
| [13] | GRANDO R B， DE JESUS J C， KICH V A， et al. Double critic deep reinforcement learning for mapless 3D navigation of unmanned aerial vehicles［J］. Journal of Intelligent & Robotic Systems， 2022， 104（2）： 29. |
| [14] | LI X， WANG X G， ZHOU H Y. Entry guidance for spatial no-fly zones avoidance via model-based reinforcement learning［J］. Aerospace Science and Technology， 2024， 153： 109405. |
| [15] | SUN B， VAN KAMPEN E J. Reinforcement-learning-based adaptive optimal flight control with output feedback and input constraints［J］. Journal of Guidance， Control， and Dynamics， 2021， 44（9）： 1685-1691. |
| [16] | 路坤锋， 贾晨辉， 黄旭， 等. 面向变构型飞行器的强化学习位置姿态一体化控制方法［J］. 宇航学报， 2024， 45（7）： 1100-1110. |
| LU K F， JIA C H， HUANG X， et al. Reinforcement learning-based integrated position and attitude control method towards morphing flight vehicles［J］. Journal of Astronautics， 2024， 45（7）： 1100-1110 （in Chinese）. | |
| [17] | 何湘远， 尘军， 郭昊， 等. 基于深度强化学习的高速飞行器攻防博弈［J］. 航天控制， 2022， 40（4）： 76-83. |
| HE X Y， CHEN J， GUO H， et al. Attack-defense game based on deep reinforcement learning for high speed vehicle［J］. Aerospace Control， 2022， 40（4）： 76-83 （in Chinese）. | |
| [18] | JIANG L， NAN Y， ZHANG Y， et al. Anti-interception guidance for hypersonic glide vehicle： A deep reinforcement learning approach［J］. Aerospace， 2022， 9（8）： 424. |
| [19] | GUO Y H， JIANG Z J， HUANG H Q， et al. Intelligent maneuver strategy for a hypersonic pursuit-evasion game based on deep reinforcement learning［J］. Aerospace， 2023， 10（9）： 783. |
| [20] | LI X， WANG X G， ZHOU H Y， et al. A novel evasion guidance for hypersonic morphing vehicle via intelligent maneuver strategy［J］. Chinese Journal of Aeronautics， 2024， 37（5）： 441-461. |
| [21] | ZHAO W S， QUERALTA J P， WESTERLUND T. Sim-to-real transfer in deep reinforcement learning for robotics： A survey［C］∥2020 IEEE Symposium Series on Computational Intelligence （SSCI）. Piscataway： IEEE Press， 2021. |
| [22] | 倪浩， 章胜， 刘福炜， 等. 基于域随机化增强EfficientZero的无人机空战智能决策［J］. 控制与决策， 2025， 40（11）： 3273-3286. |
| NI H， ZHANG S， LIU F W， et al. UAV air combat intelligent decision-making based on domain randomization enhanced EfficientZero［J］. Control and Decision， 2025， 40（11）： 3273-3286 （in Chinese）. | |
| [23] | 裴培， 何绍溟， 王江， 等. 一种深度强化学习制导控制一体化算法［J］. 宇航学报， 2021， 42（10）： 1293-1304. |
| PEI P， HE S M， WANG J， et al. Integrated guidance and control for missile using deep reinforcement learning［J］. Journal of Astronautics， 2021， 42（10）： 1293-1304 （in Chinese）. | |
| [24] | WANG J W， ZHANG R. Terminal guidance for a hypersonic vehicle with impact time control［J］. Journal of Guidance， Control， and Dynamics， 2018， 41（8）： 1790-1798. |
| [25] | 孙勇， 段广仁， 张卯瑞， 等. 高超声速飞行器再入过程改进气动系数模型［J］. 系统工程与电子技术， 2011， 33（1）： 134-137. |
| SUN Y， DUAN G R， ZHANG M R， et al. Modified aerodynamic coefficient models of hypersonic vehicle in reentry phase［J］. Systems Engineering and Electronics， 2011， 33（1）： 134-137 （in Chinese）. | |
| [26] | 高峰， 唐胜景， 师娇， 等. 一种基于落角约束的偏置比例导引律［J］. 北京理工大学学报， 2014， 34（3）： 277-282. |
| GAO F， TANG S J， SHI J， et al. A bias proportional navigation guidance law based on terminal impact angle constraint［J］. Transactions of Beijing Institute of Technology， 2014， 34（3）： 277-282 （in Chinese）. | |
| [27] | ALTMAN E. Constrained Markov decision processes： Stochastic modeling［D］. New York： Routledge， 1999. |
| [28] | SCHULMAN J， WOLSKI F， DHARIWAL P， et al. Proximal policy optimization algorithms［DB/OL］. arXiv： 1707.06347； 2017. |
| [29] | 法尔汉·A·法拉奇. 微分博弈论在导弹和自主系统制导中的应用［M］. 北京： 北京理工大学出版社， 2018： 121-146. |
| FARHAN A F. Differential game theory with applications to missiles and autonomous systems guidance［M］. Beijing： Beijing Institute of Technology Press， 2018： 121-146 （in Chinese）. |
/
| 〈 |
|
〉 |
CJA