导航

Acta Aeronautica et Astronautica Sinica ›› 2024, Vol. 45 ›› Issue (18): 329922.doi: 10.7527/S1000-6893.2023.29922

• Electronics and Electrical Engineering and Control • Previous Articles    

Decision-making method for air combat maneuver based on explainable reinforcement learning

Shuheng YANG1,2, Dong ZHANG1,2(), Wei XIONG1,2, Zhi REN1,2, Shuo TANG1,2   

  1. 1.School of Astronautics,Northwestern Polytechnical University,Xi’an 710072,China
    2.Shaanxi Key Laboratory of Aerospace Flight Vehicle Design,Northwestern Polytechnical University,Xi’an 710072,China
  • Received:2023-11-28 Revised:2024-01-10 Accepted:2024-04-07 Online:2024-04-12 Published:2024-04-12
  • Contact: Dong ZHANG E-mail:zhangdong@nwpu.edu.cn
  • Supported by:
    Collective Intelligence & Collaboration Laboratory(QXZ23013402)

Abstract:

Intelligent air combat is the trend of air combat in the future, and deep reinforcement learning is an important technical way to realize intelligent decision-making in air combat. However, due to the characteristic of “black box model”, deep reinforcement learning has the shortcomings such as difficulty in explaining strategies, understanding intentions, and trusting decisions, which brings challenges to the application of deep reinforcement learning in intelligent air combat. To solve these problems, an intelligent air combat maneuver decision-making method is proposed based on explainable reinforcement learning. Firstly, based on the strategy-level explanation method and dynamic Bayesian network, an interpretability model and the maneuvering intention recognition model are constructed. Secondly, through calculation of the importance of the decision and the probability of maneuvering intention, the intention-level of the Unmanned Aerial Vehicle (UAV) maneuver decision-making process can be explained. Finally, based on the intent interpretation results, the reward function and training strategy of the deep reinforcement learning algorithm are modified, and the effectiveness of the proposed method is verified by simulation and comparative analysis. The proposed method can obtain air combat maneuver strategies with excellent effectiveness, strong reliability, and high credibility.

Key words: intelligent air combat, reinforcement learning, maneuver decision-making, explainability, identification of air combat intention

CLC Number: