航空学报 > 2023, Vol. 44 Issue (10): 128094-128094   doi: 10.7527/S1000-6893.2023.28094

基于深度强化学习的空战机动决策试验

章胜1, 周攀1, 何扬1, 黄江涛1(), 刘刚2, 唐骥罡1, 贾怀智3, 杜昕1   

  1. 1.中国空气动力研究与发展中心 空天技术研究所,绵阳 621000 2.中国空气动力研究与发展中心,绵阳 621000
    3.西北工业大学 航空学院,西安 710000
  • 收稿日期:2022-10-08 修回日期:2023-01-05 接受日期:2023-02-15 出版日期:2023-02-27 发布日期:2023-02-24
  • 通讯作者: 黄江涛 E-mail:hjtcyf@163.com
  • 基金资助:
    国家自然科学基金(11902332)

Air combat maneuver decision-making test based on deep reinforcement learning

Sheng ZHANG1, Pan ZHOU1, Yang HE1, Jiangtao HUANG1(), Gang LIU2, Jigang TANG1, Huaizhi JIA3, Xin DU1   

  1. 1.Aerospace Technology Institute,China Aerodynamics Research and Development Center,Mianyang 621000,China
    2.China Aerodynamics Research and Development Center,Mianyang 621000,China
    3.School of Aeronautics,Northwestern Polytechnical University,Xi’an 710000,China
  • Received:2022-10-08 Revised:2023-01-05 Accepted:2023-02-15 Online:2023-02-27 Published:2023-02-24
  • Contact: Jiangtao HUANG E-mail:hjtcyf@163.com
  • Supported by:
    National Natural Science Foundation of China(11902332)

摘要:

空战智能决策将极大改变未来战争的形态与模式。深度强化学习决策机可以挖掘飞行器潜力,是实现空战智能决策的重要技术范式,但其工程实现鲜有报道。针对基于深度强化学习的双机近距空战机动智能决策的工程实现问题,开发了适于应用的深度神经网络在线机动决策模型,发展了通过飞行控制律跟踪航迹导引决策指令的机动控制方案,并进一步开展了软硬件实现工作与人机对抗飞行试验,实现了智能空战从虚拟仿真到真实飞行的迁移。研究结果表明基于本文发展的近距空战机动决策及控制方法,智能无人机在与人类“飞行员”的对抗中能够迅速做出有利于己方的动作决策,通过机动快速占据态势优势。研究结果显示了深度神经网络智能决策技术在空战决策中的潜在应用价值。

关键词: 近距空战, 智能决策, 深度强化学习, 人机对抗, 飞行试验

Abstract:

The air combat intelligent decision-making will greatly change the form of wars. Deep reinforcement learning decision-making machine, as an important technical paradigm to realize the intelligent decision-making in air combat, can explore the potential of unmanned aircraft. However, reports on its engineering implementation are rare. Aimed at the practical implementation of the maneuver intelligent decision-making based on deep reinforcement learning in the one-to-one fighters’ close-range air combat, an online deep neural network maneuver decision-making model suitable for application is developed. The maneuver control scheme that the trajectory guidance decision-making commands are tracked with the flight control law is proposed. The corresponding software and hardware architectures are realized and the human-machine combat flight test is carried out, which achieves the transfer from virtual simulation to real flight in intelligent air combat. The research results show that, based on the close-range air combat maneuver decision-making and control method developed in this paper, the intelligent unmanned aircraft can make logical maneuver decisions quickly in favor of its own side and thus is soon in the advantageous situation by maneuver when combatting with human “pilots”. The flight test results demonstrate the potential application value of the deep neural network intelligent decision-making machine in air combat decision-making.

Key words: close-range air combat, intelligent decision-making, deep reinforcement learning, human-machine combat, flight test

中图分类号: