电子电气工程与控制

针对集群攻击的飞行器智能协同拦截策略

  • 高树一 ,
  • 林德福 ,
  • 郑多 ,
  • 胡馨予
展开
  • 1.北京理工大学 宇航学院,北京 100081
    2.北京理工大学 徐特立学院,北京 100081
.E-mail: zhengduohello@126.com

收稿日期: 2022-11-23

  修回日期: 2022-12-20

  录用日期: 2023-02-22

  网络出版日期: 2023-03-03

基金资助

国家自然科学基金青年基金项目(61903350);教育部产学研创新项目(2021ZYA02002);北京理工大学青年教师学术启动计划(3010011182130)

Intelligent cooperative interception strategy of aircraft against cluster attack

  • Shuyi GAO ,
  • Defu LIN ,
  • Duo ZHENG ,
  • Xinyu HU
Expand
  • 1.School of Aerospace Engineering,Beijing Institute of Technology,Beijing 100081,China
    2.XUTELI School,Beijing Institute of Technology,Beijing 100081,China

Received date: 2022-11-23

  Revised date: 2022-12-20

  Accepted date: 2023-02-22

  Online published: 2023-03-03

Supported by

National Natural Science Foundation of China(61903350);Ministry of Education's industry-university-researchinnovation project(2021ZYA02002);Beijing Institute of Technology Research Fund Program for Young Scholars(3010011182130)

摘要

无人集群间拦截博弈对抗是未来智能化战争的重要作战场景。针对飞行器集群攻击的协同拦截博弈对抗问题,提出了一种基于近端策略优化方法的多智能体深度强化学习协同拦截策略,将单智能体近端策略优化算法和集中式评价分布式执行算法架构相结合,设计了一种多智能体强化学习智能机动策略,在此基础上为解决算法收敛慢的问题,引入广义优势函数提升算法的收敛性能。仿真结果表明,多机智能协同拦截策略赋予飞行器自主学习的属性,能够根据实时战场态势智能自主分配拦截任务,且通过约束策略更新幅度提升了算法收敛速率。经过不断迭代自学习,能够实现拦截策略的自主优化,在不同的场景下自学习提升协同拦截效能。

本文引用格式

高树一 , 林德福 , 郑多 , 胡馨予 . 针对集群攻击的飞行器智能协同拦截策略[J]. 航空学报, 2023 , 44(18) : 328301 -328301 . DOI: 10.7527/S1000-6893.2023.28301

Abstract

The attack defense confrontation and interception between unmanned clusters is an important operational scenario in the future intelligent war. Aiming at the problem of cooperative interception of game confrontation against aircraft cluster attacks, a multi-agent deep reinforcement learning cooperative interception strategy based on the near end strategy optimization method is proposed. Combining the single agent near end strategy optimization algorithm with the centralized evaluation distributed execution algorithm architecture, a multi-agent reinforcement learning intelligent maneuver strategy is designed. On this basis, to solve the problem of slow algorithm convergence, the generalized dominance function is introduced to improve the convergence performance of the algorithm. Simulation results show that the multi aircraft intelligent cooperative interception strategy endows the UAV with the attribute of autonomous learning, which can intelligently and autonomously assign interception tasks according to the real-time battlefield situation, and improves the algorithm convergence rate by constraining the update range of the strategy. Through continuous iterative self-learning, this strategy can realize the autonomous optimization of game interception strategy. Improve collaborative interception efficiency by self-learning in different scenarios.

参考文献

1 GUO D, LIANG Z X, JIANG P, et al. Weapon-target assignment for multi-to-multi interception with grouping constraint[J]. IEEE Access20197: 34838-34849.
2 GUO J G, HU G J, GUO Z Y, et al. Evaluation model, intelligent assignment, and cooperative interception in multimissile and multitarget engagement[J]. IEEE Transactions on Aerospace and Electronic Systems202258(4): 3104-3115.
3 KHOSRAVI M, AGHDAM A G. Cooperative receding horizon control for multi-target interception in uncertain environments[C]∥ 53rd IEEE Conference on Decision and Control. Piscataway: IEEE Press, 2015: 4497-4502.
4 MENG X Q, SUN B, ZHU D Q. Harbour protection: Moving invasion target interception for multi-AUV based on prediction planning interception method[J]. Ocean Engineering2021219: 108268.
5 SUN Z Y, YANG J Y. Multi-missile interception for multi-targets: Dynamic situation assessment, target allocation and cooperative interception in groups[J]. Journal of the Franklin Institute2022359(12): 5991-6022.
6 ZHU R, SUN D, ZHOU Z Y. Cooperation strategy of unmanned air vehicles for multitarget interception[J]. Journal of Guidance, Control, and Dynamics200528(5): 1068-1072.
7 JEON I S, LEE J I, TAHK M J. Impact-time-control guidance law for anti-ship missiles[J]. IEEE Transactions on Control Systems Technology200614(2): 260-266.
8 吕腾,吕跃勇,李传江,等.带空间协同的多导弹时间协同制导律[J].航空学报201839(10):322115.
  LYU T, LYU Y Y, LI C J, et al. Time cooperative guidance law for multiple missiles with space coopera-tion [J]. Acta Aeronautica et Astronautica Sinica201839(10): 322115 (in Chinese).
9 SINHA A, KUMAR S R. Supertwisting control-based cooperative salvo guidance using leader-follower approach[J]. IEEE Transactions on Aerospace and Electronic Systems202056(5): 3556-3565.
10 ZHANG P, ZHANG X Y. Multiple missiles fixed-time cooperative guidance without measuring radial velocity for maneuvering targets interception[J]. ISA Transactions2022126: 388-397.
11 SHAFERMAN V, SHIMA T. Linear quadratic guidance laws for imposing a terminal intercept angle[J]. Journal of Guidance, Control, and Dynamics200831(5): 1400-1412.
12 SUN X J, ZHOU R, HOU D L, et al. Consensus of leader-followers system of multi-missile with time-delays and switching topologies[J]. Optik2014125(3): 1202-1208.
13 ERER K, MERTTOP?UOGLU O. Indirect control of impact angle against stationary targets using biased PPN: AIAA-2010-8184[R]. Reston: AIAA, 2010.
14 HARL N, BALAKRISHNAN S N. Impact time and angle guidance with sliding mode control[J]. IEEE Transactions on Control Systems Technology201220(6): 1436-1449.
15 KUMAR S R, RAO S, GHOSE D. Nonsingular terminal sliding mode guidance with impact angle constraints[J]. Journal of Guidance, Control, and Dynamics201437(4): 1114-1130.
16 DONG X F, REN Z. Impact angle constrained distributed cooperative guidance against maneuvering targets with undirected communication topologies[J]. IEEE Access20208: 117867-117876.
17 KANG S, KIM H J. Differential game missile guidance with impact angle and time constraints[J]. IFAC Proceedings Volumes201144(1): 3920-3925.
18 WANG B L, LI S G, GAO X Z, et al. UAV swarm confrontation using hierarchical multiagent reinforcement learning[J]. International Journal of Aerospace Engineering20212021: 1-12.
19 陈灿, 莫雳, 郑多, 等. 非对称机动能力多无人机智能协同攻防对抗[J]. 航空学报202041(12): 324152.
  CHEN C, MO L, ZHENG D, et al. Cooperative attack-defense game of multiple UAVs with asymmetric maneuverability[J]. Acta Aeronautica et Astronautica Sinica202041(12): 324152 (in Chinese).
20 IMADO F, KURODA T. Family of local solutions in a missile-aircraft differential game[J]. Journal of Guidance, Control, and Dynamics201134(2): 583-591.
21 Bowling M, Veloso M. Rational and convergent learning in stochastic games[C]∥ International Joint Conference On Artificial Intelligence. Hillsdale: Lawrence Erlbaum Associates Ltd, 2001: 1021-1026.
22 罗德林, 段海滨, 吴顺详, 等. 基于启发式蚁群算法的协同多目标攻击空战决策研究[J]. 航空学报200627(6): 1166-1170.
  LUO D L, DUAN H B, WU S X, et al. Research on air combat decision-making for cooperative multiple target attack using heuristic ant colony algorithm[J]. Acta Aeronautica et Astronautica Sinica200627(6): 1166-1170 (in Chinese).
23 裴培, 何绍溟, 王江, 等. 一种深度强化学习制导控制一体化算法[J]. 宇航学报202142(10): 1293-1304.
  PEI P, HE S M, WANG J, et al. Integrated guidance and control for missile using deep reinforcement learning[J]. Journal of Astronautics202142(10): 1293-1304 (in Chinese).
24 LEE S M, KIM H, MYUNG H, et al. Cooperative coevolutionary algorithm-based model predictive control guaranteeing stability of multirobot formation[J]. IEEE Transactions on Control Systems Technology201523(1): 37-51.
25 WU X, LIU Y, XIE S R, et al. Collaborative defense with multiple USVs and UAVs based on swarm intelligence[J]. Journal of Shanghai Jiaotong University (Science)202025(1): 51-56.
26 LUO Y X, SONG J A, ZHAO K, et al. UAV-cooperative penetration dynamic-tracking interceptor method based on DDPG[J]. Applied Sciences202212(3): 1618.
文章导航

/