首页 >

基于可解释性强化学习的空战机动决策方法

杨书恒1,张栋1,熊威1,2,任智1,2,唐硕1   

  1. 1. 西北工业大学
    2. 西北工业大学 空天飞行器设计陕西省重点实验室
  • 收稿日期:2023-11-28 修回日期:2024-04-08 出版日期:2024-04-10 发布日期:2024-04-10
  • 通讯作者: 张栋
  • 基金资助:
    群体协同与自主实验室开放基金

A Decision-making Method for Air Combat Maneuver Based on Explainable Reinforcement Learning

  • Received:2023-11-28 Revised:2024-04-08 Online:2024-04-10 Published:2024-04-10
  • Supported by:
    Funded by Collective Intelligence & Collaboration Laboratory

摘要: 智能空战是未来空战的趋势,深度强化学习是实现空战智能决策的一条重要技术途径。然而由于深度强化学习的“黑箱模型”特质,存在策略难解释,意图难理解,决策难信任的缺点,给深度强化学习在智能空战中的应用带来了挑战。针对这些问题,提出了一种基于可解释性强化学习的智能空战机动动作决策方法。首先,基于策略级解释方法和动态贝叶斯网络构建了可解释性模型和机动意图识别模型;其次,通过决策重要性的计算以及机动意图概率实现无人机机动决策过程的意图层面可解释;最后,基于意图解释结果对深度强化学习算法的奖励函数和训练策略进行修正,并通过仿真对比分析验证了该设计方法的有效性。该方法能够获得有效性优、可靠性强、可信度高的空战机动策略。

关键词: 智能空战, 强化学习, 机动动作决策, 可解释性, 空战意图识别

Abstract: Intelligent air combat is the trend of air combat in the future, and deep reinforcement learning is an important technical way to realize intelligent decision-making in air combat. However, due to the "black box model" characteristics of deep reinforcement learning, there are shortcomings such as difficult to explain strategies, difficult to understand intentions, and difficult to trust decisions, which brings challenges to the application of deep reinforcement learning in intelligent air combat. To solve these problems, an intelligent air combat maneuver action decision-making method based on explainable reinforcement learning was proposed. Firstly, based on the strategy-level explainable method and dynamic Bayesian network, the interpretability model and the maneuvering intention recogni-tion model were constructed. Secondly, through the calculation of the importance of the decision and the probability of maneuvering intention, the intention level of the UAV maneuvering decision-making process can be explained. Finally, based on the intent inter-pretation results, the reward function and training strategy of the deep reinforcement learning algorithm are modified, and the effec-tiveness of the design method is verified by simulation and comparative analysis. This method can obtain air combat maneuver strat-egies with excellent effectiveness, strong reliability, and high credibility.

Key words: Intelligent air combat, Reinforcement learning, Maneuver decision-making, Explainable, Identification of air combat intentions

中图分类号: