非完备信息下无人机近距博弈自主决策

doi:10.7527/S1000-6893.2025.32215

Abstract

Abstract:

With the development of computer science， automatic control theory， aircraft design and other disciplines， autonomous decision-making of Unmanned Aerial Vehicle （UAV） in close-range game has become one of the key technical problems in the field of UAV. Aimed at the autonomous decision-making problem of UAV in close-range game under incomplete information， this paper proposes an autonomous decision-making method of UAV in close-range game based on pre-trained EfficientZero algorithm. Firstly， a three-degree-of-freedom dynamic model of UAV based on quaternion theory is implemented， and a three-degree-of-freedom close-range game environment model of UAV is established according to this method. Secondly， based on deep neural network， an autonomous decision-making model of UAV close-range game for multi-dimensional continuous state input and multi-dimensional discrete action output is established. On this basis， an optimization method of close-range game decision model based on pre-trained EfficientZero algorithm is proposed. Then， the prediction model of target maneuvering trajectory under incomplete information is established. Finally， the close-range game simulation experiment of UAV is carried out.

Key words: unmanned aerial vehicles, imperfect information, autonomous decision-making, situation prediction, artificial intelligence

CLC Number:

V249.12

Pan ZHOU, Ni LI, Jiangtao HUANG, Qinglin YANG, Yunxiao LIAN. Autonomous decision-making in close-range game under imperfect information for unmanned aerial vehicles[J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(S1): 732215.

Figures/Tables 19

Fig.1

Table 1

Values range of control variables

名称	取值范围
迎角/（°）	$- 10,30$
滚转角/（°）	$- 80,80$
油门	$0.2,1 T m a x$

Table 1

Fig.2

Table 2

Initial state of confrontation

状态变量	红机初始值	蓝机初始值
$x / m$	$0$	$- 5 000,5 000$
$y / m$	$0$	$- 5 000,5 000$
$z / m$	$2 000,10 000$	$2 000,10 000$
$v / (m · s - 1)$	$80,200$	$80,200$
$χ / r a d$	$- π, π$	$- π, π$
$γ / r a d$	$0$	$0$

Table 2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

Fig.11

Fig.12

Fig.13

Fig.14

Fig.15

Fig.16

Fig.17

References 23

[1]	杨伟. 关于未来战斗机发展的若干讨论［J］. 航空学报， 2020， 41（6）： 524377.
	YANG W. Development of future fighters［J］. Acta Aeronautica et Astronautica Sinica， 2020， 41（6）： 524377 （in Chinese）.
[2]	ARTHUR H M. Counter-drone systems［R］. 2nd Edition. Center for the Study of the Drone， 2019.
[3]	孙昭，何广军，李广剑. 美军反无人机技术研究［J］. 飞航导弹， 2021（11）： 12-18.
	SUN Z， HE G J， LI G J. Research on US army’s anti-UAV technology［J］. Aerodynamic Missile Journal， 2021（11）： 12-18 （in Chinese）.
[4]	王宇，陈浩，黄健. 有人机/无人机协同系统研究现状与展望［C］∥. 第十届中国指挥控制大会论文集（上册）. 北京：兵器工业出版社，2022： 12-17.
	WANG Y， CHEN H， HUANG J. Research status and prospects of collaborative systems between drones and aerial vehicles ［C］∥. Proceedings of the 10th China Command and Control Conference （Volume 1）.Beijing：The Publishing House of Ordnance Industry， 2022： 12-17 （in Chinese）.
[5]	严锐驰，李帅，王晨，等. 基于自博弈强化学习的异构无人机集群协同对抗决策方法［J］. 中国科学：信息科学， 2024， 54（7）： 1709-1729.
	YAN R C， LI S， WANG C， et al. Cooperative decision-making for heterogeneous UAV swarm confrontation based on self-play reinforcement learning［J］. Scientia Sinica （Informationis）， 2024， 54（7）： 1709-1729 （in Chinese）.
[6]	HINTON G E， OSINDERO S， TEH Y W. A fast learning algorithm for deep belief nets［J］. Neural Computation， 2006， 18（7）： 1527-1554.
[7]	SUTTON R S， BARTO A G. Reinforcement learning： An introduction［M］. 2nd Ed. London： MIT Press， 2018.
[8]	NGUYEN N D， NGUYEN T， NAHAVANDI S. System design perspective for human-level agents using deep reinforcement learning： A survey［J］. IEEE Access， 2017， 5： 27091-27102.
[9]	POPE A P， IDE J S， MIĆOVIĆ D， et al. Hierarchical reinforcement learning for air combat at DARPA’s AlphaDogfight trials［J］. IEEE Transactions on Artificial Intelligence， 2022， 4（6）： 1371-1385.
[10]	孟光磊，刘德见，周铭哲，等. 近距空战训练中的智能虚拟对手决策与导引方法［J］. 北京航空航天大学学报， 2022， 48（6）： 937-949.
	MENG G L， LIU D J， ZHOU M Z， et al. Intelligent virtual opponent decision making and guidance method in short-range air combat training［J］. Journal of Beijing University of Aeronautics and Astronautics， 2022， 48（6）： 937-949 （in Chinese）.
[11]	LIU P， MA Y F. A deep reinforcement learning based intelligent decision method for UCAV air combat［M］∥ Modeling， Design and Simulation of Systems. Singapore： Springer Singapore， 2017： 274-286.
[12]	YANG Q M， ZHANG J D， SHI G Q， et al. Maneuver decision of UAV in short-range air combat based on deep reinforcement learning［J］. IEEE Access， 2019， 8： 363-378.
[13]	周攀，黄江涛，章胜，等. 基于深度强化学习的智能空战决策与仿真［J］. 航空学报， 2023， 44（4）： 126731.
	ZHOU P， HUANG J T， ZHANG S， et al. Intelligent air combat decision making and simulation based on deep reinforcement learning［J］. Acta Aeronautica et Astronautica Sinica， 2023， 44（4）： 126731 （in Chinese）.
[14]	XIE L， DING D L， WEI Z L， et al. Moving time UCAV maneuver decision based on the dynamic relational weight algorithm and trajectory prediction［J］. Mathematical Problems in Engineering， 2021， 2021（1）： 6641567.
[15]	王宝来，高显忠，谢涛，等. 基于强化学习与种群博弈的近距空战决策［J］. 航空学报， 2024， 45（12）： 329466.
	WANG B L， GAO X Z， XIE T， et al. Decision-making in close-range air combat based on reinforcement learning and population game［J］. Acta Aeronautica et Astronautica Sinica， 2024， 45（12）： 329466 （in Chinese）.
[16]	李恒晖，林前辉，韩涛锋，等. 基于能量机动的近距空战模型及应用［J］. 航空学报， 2025， 46（7）： 330863.
	LI H H， LIN Q H， HAN T F， et al. Close-range air combat model based on energy maneuverability and its applications［J］. Acta Aeronautica et Astronautica Sinica， 2025， 46（7）： 330863 （in Chinese）.
[17]	杨书恒，张栋，熊威，等. 基于可解释性强化学习的空战机动决策方法［J］. 航空学报， 2024， 45（18）： 329922.
	YANG S H， ZHANG D， XIONG W， et al. Decision-making method for air combat maneuver based on explainable reinforcement learning［J］. Acta Aeronautica et Astronautica Sinica， 2024， 45（18）： 329922 （in Chinese）.
[18]	孙智孝，杨晟琦，朴海音，等. 未来智能空战发展综述［J］. 航空学报， 2021， 42（8）： 525799.
	SUN Z X， YANG S Q， PIAO H Y， et al. A survey of air combat artificial intelligence［J］. Acta Aeronautica et Astronautica Sinica， 2021， 42（8）： 525799 （in Chinese）.
[19]	ZAMBALDI V， RAPOSO D， SANTORO A， et al. Relational deep reinforcement learning［J］. arXiv preprint， arXiv：， 2018.
[20]	YE W， LIU S， KURUTACH T， et al. Mastering atari games with limited data［J］. Advances in Neural Information Processing Systems， 2021， 34： 25476-25488.
[21]	GRAVES A. Long short-term memory［M］∥Supervised Sequence Labelling with Recurrent Neural Networks. Berlin： Springer， 2012： 37-45.
[22]	HOCHREITER S， SCHMIDHUBER J. Long short-term memory［J］. Neural Computation， 1997， 9（8）： 1735-1780.
[23]	VAN HOUDT G， MOSQUERA C， NÁPOLES G. A review on the long short-term memory model［J］. Artificial Intelligence Review， 2020， 53（8）： 5929-5955.

Autonomous decision-making in close-range game under imperfect information for unmanned aerial vehicles

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 19

References 23

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	Guanghui WU, Jing WANG, Hairun XIE, Tuliang MA, Qiang MIAO, Jixin XIANG, Miao ZHANG. Data and knowledge-enabled intelligent aerodynamic design for civil aircraft [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(5): 531485-531485.
[2]	Yifeng WANG, Yiming PENG, Long LI, Xiaohui WEI, Hong NIE. DQN-based active arrest and recovery technique for UAVs [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(12): 231448-231448.
[3]	Tao CHEN, Jian CHEN. Learning-observer-based resilient fault-tolerant control for quadrotor unmanned aerial vehicles [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(11): 531346-531346.
[4]	Yongnan JIA. A scheme for unmanned aerial system traffic management in low-altitude airspace [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(11): 531399-531399.
[5]	Shusheng CHEN, Muliang JIA, Jiahao LIN, Shiyi JIN, Zhenghong GAO, Yueqing WANG, Zhiqiang MA, Zheng LI, Chenlong DUAN, Jiawei LI. Empowering aircraft technology applications with generative models: Research progress and prospects [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(10): 631194-631194.
[6]	Chunhui ZHAO, Anmeng LIU, Yang LYU, Quan PAN. A survey of resilient self-localization for UAV [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(8): 28839-028839.
[7]	Shengzhe SHAN, Weiwei ZHANG. Air combat intelligent decision-making method based on self-play and deep reinforcement learning [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(4): 328723-328723.
[8]	Junshen LI, Xiangyan MENG, Nuannuan SHI, Wei LI, Ninghua ZHU, Ming LI. Intelligent processing of optical neural networks: Technological evolution and future prospects [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(20): 630439-630439.
[9]	Yuexing WANG, Qiyang ZHOU, Minqian LI, Weixing MIAO, Yandi LU, Xianliang GE. Prototyping of UAV swarm control software [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(20): 630678-630678.
[10]	Qingrui ZHANG, Yunyun LIU, Huijie SUN, Bo ZHU. Robust cooperative tracking control for close formation of fixed⁃wing unmanned aerial vehicles [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(1): 629233-629233.
[11]	An ZHANG, Mi YANG, Wenhao BI, Baichuan ZHANG, Yunong WANG. Task allocation of heterogeneous multi-UAVs in uncertain environment based on multi-strategy integrated GWO [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2023, 44(8): 327115-327115.
[12]	Linkun HE, Wenchao XUE, Ran ZHANG, Huifeng LI. Guidance and control for powered descent and landing of launch vehicles: Overview and outlook [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(23): 628462-628462.
[13]	Yuwei LIU, Yuqiang CHENG, Jianjun WU. Research progress of intelligent control methods in space propulsion systems [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(15): 528505-528505.
[14]	HU Yangxiu, ZHAO Changchun, JIA Chenglong, QIAN Zhouyuan, HU Tao. Synchronous path formation control of UAV swarm based on robot operating system (ROS) [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(S1): 726914-726914.
[15]	LIU Fang, HAN Xiao. Adaptive aerial object detection based on multi-scale deep learning [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(5): 325270-325270.