战斗机嵌入式训练系统中的智能虚拟陪练

陈斌; 王江; 王阳

doi:10.7527/S1000-6893.2019.23467

航空学报 >

2020 , Vol. 41 >Issue 6: 523467 - 523467

DOI: https://doi.org/10.7527/S1000-6893.2019.23467

电子电气工程与控制

战斗机嵌入式训练系统中的智能虚拟陪练

陈斌 ,
王江 ,
王阳

展开

中国航空工业成都飞机设计研究所, 成都 610074

收稿日期: 2019-09-08

修回日期: 2019-10-08

网络出版日期: 2019-11-14

收起

Intelligent virtual training partner in embedded training system of fighter

CHEN Bin ,
WANG Jiang ,
WANG Yang

Expand

AVIC Chengdu Aircraft Design and Research Institute, Chengdu 610074, China

Received date: 2019-09-08

Revised date: 2019-10-08

Online published: 2019-11-14

Fold

摘要

智能化"实虚"对抗是现代先进战斗机嵌入式训练系统的重要功能需求。自主空战决策控制技术在未来空战装备发展中扮演关键角色。将当前的功能需求和发展中的技术结合起来，得到了空战智能虚拟陪练的概念。先进控制决策技术的引入使得智能虚拟陪练能够帮助飞行员完成复杂的战术训练，而训练中真实的对抗场景为技术的验证提供了理想的环境，大量的训练数据为技术的持续迭代优化提供了保障。作为可学习和进化的空战战术专家，智能陪练在人机对抗和自我对抗中不断优化，当其具备与人相当甚至超越人的战术能力时，可应用于未来的无人空战系统。智能虚拟陪练需要具备4项基本能力：智能决策能力、知识学习能力、对抗自优化能力和参数化表示能力。对其包含的关键技术进行了分析，提出并实现了一个基于模糊推理、神经网络和强化学习的解决方案，展示了其各项基本能力及目前达到的空战水平。未来更多的模型和算法可在智能虚拟陪练的框架中进行验证和优化。

关键词： 嵌入式训练系统; 智能虚拟陪练; 自主空战; 模糊推理; 神经网络; 强化学习

本文引用格式

陈斌 , 王江 , 王阳 . 战斗机嵌入式训练系统中的智能虚拟陪练[J]. 航空学报, 2020 , 41(6) : 523467 -523467 . DOI: 10.7527/S1000-6893.2019.23467

Abstract

Intelligent ‘live vs. virtual’ counterwork is an important function requirement for modern advanced fighter’s embedded training system. Autonomous decision making and control technology plays a vital role in the development of future air battle equipment. The combination of the current requirement and developing technology creates the concept of air combat intelligent virtual training partner. Advanced decision-making and control technology enables the intelligent virtual training partner to help pilots fulfill complex tactics training, during which live scenarios and amount of data provide ideal validation environment and continuous optimization opportunities. As an air combat tactics expert that allows self-learning and self-evolving, the intelligent training partner is able to get consistent optimization in counterwork with pilots and itself. And it can be applied into future unmanned air battle systems when it has tactical advantages equal even superior to pilots. The four basic capability requirements for intelligent virtual training partner are presented: intelligent decision making, knowledge learning, self-countering and optimizing, and parameterization representing. The key technologies involved in these requirements are analyzed. A prototype solution is built based on fuzzy inference, neural network and reinforcement learning, and their basic capabilities and current air combat level were shown in this paper. In the future, more models and algorithms can be validated and optimized in the framework of intelligent virtual training partner.

Key words： embedded training system; intelligent virtual training partner; autonomous air combat; fuzzy inference; neural network; reinforcement learning

参考文献

[1] 亓凯,杨任农,左家亮,等. 空战飞机嵌入式训练系统的研究[J]. 火力与指挥控制, 2011, 36(9):165-171. QI K, YANG R N, ZUO J L, et al. Research on embedded training system in combat aircraft[J]. Fire Control & Command Control, 2011, 36(9):165-171(in Chinese).
[2] 耿振余, 孙金标, 李德龙,等. 机载嵌入式战术对抗训练系统设计[J]. 系统仿真学报, 2014, 26(12):2882-2886. GENG Z Y, SUN J B, LI D L, et al. Design of airborne embedded training system of air combat counterwork[J]. Journal of System Simulation, 2014, 26(12):2882-2886(in Chinese).
[3] 陈凌,吴冰,胡志伟, 等. 机载嵌入式空战训练的研究与进展[J]. 计算机仿真,2010,27(2):108-112. CHEN L, WU B, HU Z W, et al. The research and advances on airborne embedded training for air combat[J]. computer simulation, 2010, 27(2):108-112(in Chinese).
[4] 耿振余,刘思彤,李德龙. 嵌入式空战训练中虚拟智能对手的生成研究[J]. 现代防御技术,2014, 42(3):172-177(in Chinese). GENG Z Y, LIU S T, LI D L. Generating virtual intelligent adversary in embedded training of air combat counterwork[J]. Modern Defense Technology, 2014, 42(3):172-177(in Chinese).
[5] 袁坤刚, 张靖, 刘波, 等. 目标飞机自主空战战术机动仿真[J]. 中国电子科学研究院学报,2013, 8(3):295-299. YUAN K G, ZHANG J, LIU B, et al. Simulation of target-aircraft tactical maneuvers in autonomous aircombat[J]. Journal of China Academy of Electronics and Information Technology, 2013, 8(3):295-299(in Chinese).
[6] 董彦非, 阴小晖, 彭世冲. 空战仿真目标机战法实现[J]. 南昌航空大学学报(自然科学版), 2012, 26(1):61-65. DONG Y F, YIN X H, PENG S C. The realization of target aircraft combat plan in air combat simulation[J]. Journal of Nanchang Hangkong University (Nature Science), 2012, 26(1):61-65(in Chinese).
[7] 刘纯,李维,刘洁,等. 高级教练机嵌入式训练系统应用[J]. 兵器装备工程学报, 2017(4):26-31. LIU C, LI W, LIU J, et al. Application of advanced trainer embedded training system[J]. Journal of Ordnance Equipment Engineering, 2017(4):26-31(in Chinese).
[8] 吴雄,刘纯. 外军战斗机空战战术训练系统应用研究[J]. 兵器装备工程学报, 2017(7):37-43. WU X, LIU C. Research on the application of foreign fighter air combat tactical training system[J]. Journal of Ordnance Equipment Engineering, 2017(7):37-43(in Chinese).
[9] 周思羽, 吴文海, 张楠,等. 自主空战机动决策方法综述[J]. 航空计算技术, 2012,42(1):27-31. ZHOU S Y, WU W H, ZHANG N, et al. Overview of autonomous air combat maneuver decision[J]. Aeronautical Computing Technique, 2012, 42(1):27-31(in Chinese).
[10] 董彦非, 郭基联, 张恒喜. 空战机动决策方法研究[J]. 火力与指挥控制, 2002, 27(2):75-78. DONG Y F, GUO J L, ZHANG H X. The methods of air combat maneuvering decision[J]. Fire Control & Command Control, 2002, 27(2):75-78(in Chinese).
[11] 黄长强. 未来空战过程智能化关键技术研究[J]. 航空兵器, 2019, 26(1):11-19. HUANG C Q. Research on key technology of future air combat process intelligentization[J]. Aero Weaponry, 2019, 26(1):11-19(in Chinese).
[12] 孙永芹, 孙涛, 范洪达, 等. 现代空战机动决策研究[J]. 海军航空工程学院学报, 2009, 24(5):573-577. SUN Y Q, SUN T, FAN H D, et al. Research on maneuvering decision for modern air combat[J]. Journal of Naval Aeronautical and Astronautical University, 2009, 24(5):573-577(in Chinese).
[13] PAN Q, ZHOU D, HUANG J, et al. Maneuver decision for cooperative close-range air combat based on state predicted influence diagram[C]//2017 IEEE International Conference on Information and Automation (ICIA). Piscataway:IEEE Press, 2017:726-731.
[14] 钱炜祺, 车竞, 何开锋. 基于矩阵博弈的空战决策方法[C]//2014第二届中国指挥控制大会. 北京:中国指挥控制学会,2014:409-413. QIAN W Q, CHE J, HE K F. Air combat decision method based on game-matrix approach[C]//The 2nd China Conference on Command and Control. Beijing:Chinese Institute of Command and Control, 2014:409-413(in Chinese).
[15] 郭昊, 周德云, 张堃. 无人作战飞机空战自主机动决策研究[J]. 电光与控制, 2010, 17(8):28-32. GUO H, ZHOU D Y, ZHANG K. Study on UCAV autonomous air combat maneuvering decision-making[J]. Electronics Optics & Control, 2019, 17(8):28-32(in Chinese).
[16] 马耀飞, 马小乐. 一种空战智能决策方法研究[C]//2014中国制导、导航与控制学术会议,2014:2449-2454. MA Y F, MA X L.The methods of air combat intelligent decision[C]//Proceedings of 2014 IEEE Chinese Guidance, Navigation and Control Conference, 2014:2449-2454(in Chinese).
[17] 黄长强, 赵克新, 韩邦杰,等. 一种近似动态规划的无人机机动决策方法[J]. 电子与信息学报, 2018, 40(10):166-171. HUANG C Q, ZHAO K X, HAN B J, et al. Maneuvering decision-making method of UAV based on approximation dynamic programming[J]. Journal of Electronics & Information Technology, 2018, 40(10):166-171(in Chinese).
[18] MCGREW J S, HOW J P, WILLIAMS B, et al. Air-combat strategy using approximate dynamic programming[J]. Journal of Guidance, Control, and Dynamics, 2010, 33(5):1641-1654.
[19] 张立鹏, 魏瑞轩, 李霞. 无人作战飞机空战自主战术决策方法研究[J]. 电光与控制, 2012,19(2):92-96. ZHANG L P, WEI R X, LI X. Autonomous tactical decision-making of UCAVs in air combat[J]. Electronics Optics & Control, 2012, 19(2):92-96(in Chinese).
[20] 张磊. 无人作战飞机自主决策技术研究[J]. 航空科学技术, 2014,25(5):49-53. ZHANG L. Research on autonomous decision-making technology of UCAV[J]. Aeronautical Science & Technology,2014, 25(5):49-53(in Chinese).
[21] 唐传林, 黄长强, 丁达理,等. 一种UCAV自主空战智能战术决策方法[J]. 指挥控制与仿真, 2015,37(5):5-11. TANG C L, HUANG C Q, DING D L, et al. A method of intelligent tactical decision making for UCAV autonomous air combat[J]. Command Control & Simulation, 2015, 37(5):5-11(in Chinese).
[22] MA S, ZHANG H, YANG G. Target threat level assessment based on cloud model under fuzzy and uncertain conditions in air combat simulation[J]. Aerospace Science and Technology, 2017, 67:49-53.
[23] ERNEST N, COHEN K, KIVELEVITCH E. Genetic fuzzy Trees and their applications towards autonomous training and control of a squadron of unmanned combat aerial vehicles[J]. Unmanned Systems, 2015, 3(3):185-204.
[24] 孟光磊, 罗元强, 梁宵,等. 基于动态贝叶斯网络的空战决策方法[J]. 指挥控制与仿真, 2017,39(3):49-54. MENG G L, LUO Y Q, LIANG X, et al. Air combat decision-making method based on dynamic bayesian network[J]. Command Control & Simulation, 2017, 39(3):49-54(in Chinese).
[25] HUANG C Q, DONG K S, HUANG H Q, et al. Autonomous air combat maneuver decision using Bayesian inference and moving horizon optimization[J]. Journal of Systems Engineering and Electronics, 2018, 29(1):86-97.
[26] VOLODYMYR M, KORAY K, DAVID S, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540):529-533.
[27] LI H, WEI T, REN A, et al. Deep reinforcement learning:Framework, applications, and embedded implementations[C]//2017 IEEE/ACM International Conference on Computer-Aided Design(ICCAD). Piscatawy:IEEE Press, 2017:13-16.
[28] 左家亮, 杨任农, 张滢,等. 基于启发式强化学习的空战机动智能决策[J]. 航空学报, 2017,38(10):321168. ZUO J L, YANG R N, ZHANG Y, et al. Intelligent decision-making in air combat maneuvering based on heuristic reinforcement learning[J]. Acta Aeronautica et Astronautica Sinica, 2017, 38(10):321168(in Chinese).
[29] 方君, 闫文君, 邓向阳,等. 基于Q-学习和行为树的CGF空战行为决策[J]. 计算机与现代化, 2017(5):39-44. FANG J, YAN W J, DENG X Y, et al. Air bat strategies of CGF based on Q-learning and behavior tree[J]. Computer and Modernization, 2017(5):39-44(in Chinese).
[30] 张强, 杨任农, 俞利新, 等. 基于Q-network强化学习的超视距空战机动决策[J]. 空军工程大学学报(自然科学版), 2018, 19(6):12-18. ZHANG Q, YANG R N, YU L X, et al. BVR air combat maneuvering decision by using Q-network reinforcement learning[J]. Journal of Air Force Engineering University (Nature Science Edition), 2018, 19(6):12-18(in Chinese).
[31] 杜海文, 崔明朗, 韩统, 等. 基于多目标优化与强化学习的空战机动决策[J]. 北京航空航天大学学报, 2018, 44(11):4-13. DU H W, CUI M L, HAN T, et al. Maneuvering decision in air combat based on multi-objective optimization and reinforcement learning[J]. Journal of Beijing University of Aeronautics and Astronautics, 2018, 44(11):4-13(in Chinese).
[32] 毛梦月, 张安, 周鼎, 等. 基于机动预测的强化学习无人机空中格斗研究[J]. 电光与控制, 2019, 26(2):9-14. MAO M Y, ZHANG A, ZHOU D, et al. Reinforcement learning of UCAV air combat based on maneuver prediction[J]. Electronics Optics & Control, 2019, 26(2):9-14(in Chinese).
[33] 张菁, 何友, 彭应宁, 等. 基于神经网络和人工势场的协同博弈路径规划[J]. 航空学报, 2019, 40(3):322493. ZHANG J, HE Y, PENG Y N, et al. Neural network and artificial potential field based cooperative and adversarial path planning[J]. Acta Aeronautica et Astronautica Sinica, 2019, 40(3):322493(in Chinese).
[34] LUO P, XIE J, CHE W. Q-learning based air combat target assignment algorithm[C]//2016 IEEE International Conference on Systems, Man, and Cybernetics(SMC). Piscataway:IEEE Press, 2016:779-783.
[35] 黄长强, 唐上钦. 从"阿法狗"到"阿法鹰"——论无人作战飞机智能自主空战技术[J]. 指挥与控制学报, 2016, 2(3):261-264. HUANG C Q, TANG S Q. From Alphago to Alphaeagle:On the intelligent autonomous air combat technology for UCAV[J]. Journal of Command and Control, 2016, 2(3):261-264(in Chinese).
[36] 吴娜, 刁联旺. 基于机器学习的博弈对抗模型优化框架软件系统设计[C]//第六届中国指挥控制大会,2018:311-314. WU N, DIAO L W. Design of framework software System used to optimize of game antagonism model based on machine learning[C]//The 6th China Conference on Command and Control, 2018:311-314(in Chinese).
[37] 曹慧敏, 黄安祥, 雷祥. 空战临战态势评估方法研究[J]. 系统仿真学报, 2019, 31(2):95-100. CAO H M, HUANG A X, LEI X. Evaluation method of imminent battle situation in air combat[J]. Journal of System Simulation, 2019, 31(2):95-100(in Chinese).
[38] 郝志伟. 空战中的多目标威胁评估方法[J]. 弹箭与制导学报, 2016,36(1):177-181. HAO Z W. Threat assessment method of multi-target in air combat[J]. Journal of Projectiles, Rockets, Missiles and Guidance, 2016, 36(1):177-181(in Chinese).
[39] LU C, ZHOU Z, LIU H, et al. Situation assessment of far-distance attack air combat based on mixed dynamic Bayesian networks[C]//Proceedings of the 37th Chinese Control Conference, 2018:4569-4574.
[40] 李高垒, 马耀飞. 基于深度网络的空战态势特征提取[J]. 系统仿真学报, 2017,29(S1):98-105,112. LI G L, MA Y F. Feature extraction algorithm of air combat situation based on deep neural networks[J]. Journal of System Simulation, 2017,29(S1):98-105,112(in Chinese).
[41] 张彬超, 寇雅楠, 邬蒙, 等. 基于深度置信网络的近距空战态势评估[J]. 北京航空航天大学学报, 2017,43(7):1450-1459. ZHANG B C, KOU Y N, WU M, et al. Close-range air combat situation assessment using deep belief network[J]. Journal of Beijing University of Aeronautics and Astronautics, 2017, 43(7):1450-1459(in Chinese).
[42] DAVID S, AJA H, CHRIS J M, et al. Mastering the game of Go with deep neural networks and tree search[J]. Nature, 2016, 529(7587):484-489.
[43] 郑江安, 郭建奇, 龚旭东. 一对一超视距空战仿真中的机载雷达模型研究[J].系统仿真学报, 2012, 24(3):551-555. ZHENG J A, GUO J Q, GONG X D. Study on model of airborne radar in one wersus one beyond visual range air combat[J]. Journal of System Simulation, 2012, 24(3):551-555(in Chinese).
[44] TENG T H, TAN A H, TEOW L N. Adaptive computer-generated forces for simulator-based training[J]. Expert Systems with Applications, 2013, 40(18):7341-7353.
[45] 周光霞, 周方. 美军人工智能空战系统阿尔法初探[C]//第六届中国指挥控制大会论文集,2018:61-65. ZHOU G X, ZHOU F. Analysis of ALPHA AI for air-to-air combat of US[C]//The 6th China Conference on Command and Control, 2018:61-65(in Chinese).
[46] DAVID S, JULIAN S, KAREN S, et al. Matering the game of go without human knowledge[J]. Nature, 2017, 550(7676):354-359.
[47] VOLODYMYR M,ADRIÀ P B,MEHDI M, et al. Asynchronous methods for deep reinforcement learning[C]//Proceedings of the 33 rd International Conference on Machine Learning, 2016:1928-1937.
[48] ADAMSKI I, ADAMSKI R, GREL T, et al. Distributed deep reinforcement learning:Learn how to play atari games in 21 minutes[C]//Proceedings of International Conference on High Performance Computing, 2018:370-388.

Options

文章导航

模态框（Modal）标题

摘要

本文引用格式

Abstract

参考文献