多智能体多耦合任务混合式智能决策架构设计

doi:10.7527/S1000-6893.2023.29770

Abstract

Abstract:

To address the coupling problem and decision-making problem of multiple tasks such as task allocation and path planning of multi-agents in complex application scenarios， a design method of hybrid intelligent decision-making framework for multi-agent and multi-coupling tasks is proposed. Firstly， the advantages of single agent multi-task hybrid framework and multi-agent distributed collaborative control， a hybrid intelligent decision-making framework for multi-agent and multi-coupling tasks is designed. Secondly， the strategy network of the framework and the training controller for the strategy network are designed and a coupling relationship matrix based on coupling relationships is proposed to achieve efficient training of multi-agents and multi-tasks in face of collaborative decision-making problems. Finally， this paper modeled， trained algorithm and simulated in simulation environment，and compared with the tradition method to verifies the effectiveness and advantages of the proposed method.

Key words: intelligent decision, deep reinforcement learning, multi-agent, path planning, task allocation

CLC Number:

V249.1

Xuejian WANG, Yongming WEN, Xiaorong SHI, Ningning ZHANG, Jiexi LIU. Design of hybrid intelligent decision framework for multi⁃agent and multi⁃coupling tasks[J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(S2): 729770-729770.

Figures/Tables 5

References 20

1	VERMA J K， RANGA V. Multi-robot coordination analysis， taxonomy， challenges and future scope［J］. Journal of Intelligent & Robotic Systems， 2021， 102（1）： 10.
2	林萌龙，陈涛，任棒棒，等. 基于多智能体深度强化学习的体系任务分配方法［J］. 指挥与控制学报， 2023， 9（1）： 93-102.
	LIN M L， CHEN T， REN B B， et al. Task assignment method of operation system of systems based on multi- agent deep reinforcement learning［J］. Journal of Command and Control， 2023， 9（1）： 93-102 （in Chinese）.
3	TANG J， DUAN H B， LAO S Y. Swarm intelligence algorithms for multiple unmanned aerial vehicles collaboration： A comprehensive review［J］. Artificial Intelligence Review， 2023， 56（5）： 4295-4327.
4	GUO J G， HU G J， GUO Z Y， et al. Evaluation model， intelligent assignment， and cooperative interception in multimissile and multitarget engagement［J］. IEEE Transactions on Aerospace and Electronic Systems， 2022， 58（4）： 3104-3115.
5	刘庆周，吴锋. 多智能体路径规划研究进展［J］. 计算机工程， 2020， 46（4）： 1-10.
	LIU Q Z， WU F. Research progress of multi-agent path planning［J］. Computer Engineering， 2020， 46（4）： 1-10 （in Chinese）.
6	RAHMAN M， ALAM M A， ISLAM M M， et al. An adaptive agent-specific sub-optimal bounding approach for multi-agent path finding［J］. IEEE Access， 1092， 10： 22226-22237.
7	刘文兵，王艺栋. 多无人机协同搜索多目标的路径规划问题研究［J］. 电光与控制， 2019， 26（3）： 35-38， 73.
	LIU W B， WANG Y D. Path planning of multi-UAV cooperative search for multiple targets［J］. Electronics Optics & Control， 2019， 26（3）： 35-38， 73 （in Chinese）.
8	丁文俊，柴亚军，侯冬冬，等. AUV&UAV跨域协同搜索与跟踪路径规划［J］. 航空学报， 2023， 44（21）： 128471.
	DING W J， CHAI Y J， HOU D D， et al. Path planning for AUV & UAV cross-domain collaborative search and tracking［J］. Acta Aeronautica et Astronautica Sinica， 2023， 44（21）： 128471 （in Chinese）.
9	刘畅，谢文俊，张鹏，等. 多基地多无人机航迹避障任务规划［J］. 计算机工程， 2019， 45（11）： 275-280.
	LIU C， XIE W J， ZHANG P， et al. Mission planning for multi-base multi-UAV obstacle avoidance［J］. Computer Engineering， 2019， 45（11）： 275-280 （in Chinese）.
10	赵飞虎，李哲，王宁，等. 面向战场的多无人机协同打击航迹规划［J］. 电光与控制， 2023， 30（9）： 9-14， 91.
	ZHAO F H， LI Z， WANG N， et al. Path planning for multi-UAV cooperative strike in battlefield environments［J］. Electronics Optics & Control， 2023， 30（9）： 9-14， 91 （in Chinese）.
11	陈明强，冯树娟，李奇峰. 基于改进人工势场的物流无人机三维航迹规划［J］. 无线电工程， 2023， 53（10）： 2352-2359.
	CHEN M Q， FENG S J， LI Q F. Three-dimensional trajectory planning of logistics UAV based on improved artificial potential field［J］. Radio Engineering， 2023， 53（10）： 2352-2359 （in Chinese）.
12	贾高伟，王建峰. 无人机集群任务规划方法研究综述［J］. 系统工程与电子技术， 2021， 43（1）： 99-111.
	JIA G W， WANG J F. Research review of UAV swarm mission planning method［J］. Systems Engineering and Electronics， 2021， 43（1）： 99-111 （in Chinese）.
13	李璐璐，朱睿杰，隋璐瑶，等.智能集群系统的强化学习方法综述［J/OL］.计算机学报. .
	LI L L， ZHU R J， SUI L Y， et al. The reinforcement learning methods for intelligent collective system： A survey［J/OL］. Chinese Journal of Computers. .
14	马卫华. 导弹/火箭制导、导航与控制技术发展与展望［J］. 宇航学报， 2020， 41（7）： 860-867.
	MA W H. Review and prospect of missile/launch vehicle guidance， navigation and control technologies［J］. Journal of Astronautics， 2020， 41（7）： 860-867 （in Chinese）.
15	文永明，石晓荣，黄雪梅，等. 一种无人机集群对抗多耦合任务智能决策方法［J］. 宇航学报， 2021， 42（4）： 504-512.
	WEN Y M， SHI X R， HUANG X M， et al. An intelligent decision-making method for multi-coupling tasks of UAV cluster countermeasure［J］. Journal of Astronautics， 2021， 42（4）： 504-512 （in Chinese）.
16	刘全，翟建伟，章宗长，等. 深度强化学习综述［J］. 计算机学报， 2018， 41（1）： 1-27.
	LIU Q， ZHAI J W， ZHANG Z C， et al. A survey on deep reinforcement learning［J］. Chinese Journal of Computers， 2018， 41（1）： 1-27 （in Chinese）.
17	LOWE R， WU Y， TAMAR A， et al. Multi-agent actor-critic for mixed cooperative-competitive environments［C］∥ Proceedings of the 31st International Conference on Neural Information Processing Systems. New York： ACM， 2017： 6382–6393.
18	SUKHBAATAR S， SZLAM A， FERGUS R. Learning multiagent communication with backpropagation［C］∥ Proceedings of the 30th International Conference on Neural anInformation Processing Systems. New York： ACM， 2016： 2252–2260.
19	王子豪，张严心，黄志清，等. 部分可观测下基于RGMAAC算法的多智能体协同［J］. 控制与决策， 2023， 38（5）： 1267-1277.
	WANG Z H， ZHANG Y X， HUANG Z Q， et al. Multi- agent collaboration based on RGMAAC algorithm under partial observability［J］. Control and Decision， 2023， 38（5）： 1267-1277 （in Chinese）.
20	孙英博，苗国英，庄亚楠. 基于改进的深度强化学习多智能体协作方法［J］. 传感器与微系统， 2023， 42（9）： 25-29.
	SUN Y B， MIAO G Y， ZHUANG Y N. Multi-agent collaboration method based on improved deep reinforcement learning［J］. Transducer and Microsystem Technologies， 2023， 42（9）： 25-29 （in Chinese）.

[1]	Honglin ZHANG, Jianjun LUO, Weihua MA. Spacecraft game decision making for threat avoidance of space targets based on machine learning [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(8): 329136-329136.
[2]	Hongyu YIN, Yu WU, Tianjiao LIANG. Cooperative path planning for patrol coverage of fixed wing UAV [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(6): 328944-328944.
[3]	Yunpeng CAI, Dapeng ZHOU, Jiangchuan DING. Intelligent collaborative control of UAV swarms with collision avoidance safety constraints [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(5): 529683-529683.
[4]	Shengzhe SHAN, Weiwei ZHANG. Air combat intelligent decision-making method based on self-play and deep reinforcement learning [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(4): 328723-328723.
[5]	Zhu WANG, Mengtong ZHANG, Zhenpeng ZHANG, Guangtong XU. Multi-UAV cooperative path planning based on multi-index dynamic priority [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(4): 328816-328816.
[6]	Zhilin FAN, Hongyong YANG, Yilin HAN. Target round-up control for multi-agent systems based on reinforcement learning [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(S1): 727487-727487.
[7]	Lianbo YU, Pinzhao CAO, Liang SHI, Jie LIAN, Dong WANG. An improved conflict⁃based search algorithm for multi⁃agent path planning [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(S1): 727648-727648.
[8]	Yajie MA, Juan WANG, Bin JIANG, Jianye GONG. A fault⁃tolerant control scheme for UAVs-UGVs formation systems [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2023, 44(8): 327216-327216.
[9]	An ZHANG, Mi YANG, Wenhao BI, Baichuan ZHANG, Yunong WANG. Task allocation of heterogeneous multi-UAVs in uncertain environment based on multi-strategy integrated GWO [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2023, 44(8): 327115-327115.
[10]	Xiaowei FU, Zhe XU, Jindong ZHU, Nan WANG. Maneuvering decision-making of multi-UAV attack-defence confrontation based on PER-MATD3 [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2023, 44(7): 327083-327083.
[11]	Xizhen GAO, Liang TANG, Huang HUANG. Deep reinforcement learning in autonomous manipulation for celestial bodies exploration: Applications and challenges [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2023, 44(6): 26762-026762.
[12]	Yong XU, Hongtao YAN, Tao JIA, Yue MA, Zehua DENG, Duoneng LIU. Aerial simulation docking technology of fixed-wing clustering UAVs [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2023, 44(5): 326539-326539.
[13]	Pan ZHOU, Jiangtao HUANG, Sheng ZHANG, Gang LIU, Bowen SHU, Jigang TANG. Intelligent air combat decision making and simulation based on deep reinforcement learning [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2023, 44(4): 126731-126731.
[14]	Xiangwei ZHU, Dan SHEN, Kai XIAO, Yuexin MA, Xiang LIAO, Fuqiang GU, Fangwen YU, Kefu GAO, Jingnan LIU. Mechanisms, algorithms, implementation and perspectives of brain⁃inspired navigation [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(19): 28569-028569.
[15]	Shuyi GAO, Defu LIN, Duo ZHENG, Xinyu HU. Intelligent cooperative interception strategy of aircraft against cluster attack [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(18): 328301-328301.

Design of hybrid intelligent decision framework for multi⁃agent and multi⁃coupling tasks

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 5

References 20

Related Articles 15

Recommended Articles

Metrics

Comments