具有防撞安全约束的无人机集群智能协同控制

doi:10.7527/S1000-6893.2023.29683

Abstract

Abstract:

An intelligent cooperative control method for Unmanned Aerial Vehicle （UAV） swarms combining Deep Reinforcement Learning （DRL） and the collision avoidance strategy is proposed. First， the problem of UAV swarm control is described， and a motion model of the UAV is established. Secondly， under the framework of deep reinforcement learning， the training mechanism of the UAV swarm control strategy is established. By analyzing the observation space and action space of the UAV， the network structure of the UAV swarm control strategy is designed. The reward function is designed with the safety， tightness and consistency of the UAV swarm as the main goal. To solve the problem of lack of safety guarantee in the DRL method， the collision avoidance strategy of the UAV swarm is designed at the same time， and the collision avoidance commands can be calculated according to the relative motion states between UAVs and between UAVs and threat areas to avoid UAVs collision. The simulation results show that the proposed control method can effectively avoid the risk of collision between UAVs， guarantee the safety of the UAV swarm， and make the UAV swarm more compact.

Key words: UAV swarm, collaborative control, deep reinforcement learning, collision avoidance strategy, safety constraint

CLC Number:

V249.1

Yunpeng CAI, Dapeng ZHOU, Jiangchuan DING. Intelligent collaborative control of UAV swarms with collision avoidance safety constraints[J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(5): 529683-529683.

Figures/Tables 9

Fig.1

Fig. 2

Fig.3

Fig.4

Table 1

Parameter values of swarm control method

参数	取值	参数	取值
$γ$	0.99	$T m a x$	2 000
$Δ v m a x c$ / $(m ⋅ s - 1)$	5	N	5
$Δ ψ m a x c$ /rad	$π / 2$	K	128
$Δ z c$ /m	5	$τ$	0.01
$d ⌢$ /m	10	$d u s$ /m	10
$v H, m a x$ / $(m ⋅ s - 1)$	15	$d o s$ /m	50
$d m a x c$ /m	20	$Δ v H s$ / $(m ⋅ s - 1)$	10
$α$	0.2	$Δ ψ s$ /rad	15/180
C	1×10⁵	$Δ z s$ /m	10
$E m a x$	1 000

Table 1

Fig.5

Fig.6

Fig.7

Fig.8

References 24

1	刘伯健，李爱军，郭永，等. 带有输入受限的无人机精确编队合围容错控制［J］. 航空学报， 2023， 44（9）： 327414.
	LIU B J， LI A J， GUO Y， et al. Fault-tolerant containment control for precise formation of UAVs with input saturation［J］. Acta Aeronautica et Astronautica Sinica， 2023， 44（9）： 327414 （in Chinese）.
2	刘树光，刘荣华，王欢，等. 国外无人机集群协同控制技术新进展［J］. 飞航导弹， 2021（8）： 24-31.
	LIU S G， LIU R H， WANG H， et al. New progress of cooperative control technology of UAV cluster abroad［J］. Aerodynamic Missile Journal， 2021（8）： 24-31 （in Chinese）.
3	吴杰宏，李丹阳. 无人机集群编队控制方法研究综述［J］. 无线电通信技术， 2023， 49（4）： 589-596.
	WU J H， LI D Y. A review of UAV cluster formation control methods［J］. Radio Communications Technology， 2023， 49（4）： 589-596 （in Chinese）.
4	ALI Z A， ISRAR A， ALKHAMMASH E H， et al. A leader-follower formation control of multi-UAVs via an adaptive hybrid controller［J］. Complexity， 2021， 2021： 1-16.
5	CHEN Q Y， WANG Y J， LU Y F. Formation control for UAVs based on the virtual structure idea and nonlinear guidance logic［C］∥2021 6th International Conference on Automation， Control and Robotics Engineering （CACRE）. Piscataway： IEEE Press， 2021： 135-139.
6	吴宇，梁天骄. 基于改进一致性算法的无人机编队控制［J］. 航空学报， 2020， 41（9）： 323848.
	WU Y， LIANG T J. Improved consensus-based algorithm for unmanned aerial vehicle formation control［J］. Acta Aeronautica et Astronautica Sinica， 2020， 41（9）： 323848 （in Chinese）.
7	王晶，顾维博，窦立亚. 基于Leader-Follower的多无人机编队轨迹跟踪设计［J］. 航空学报， 2020， 41（S1）： 723758.
	WANG J， GU W B， DOU L Y. Leader-Follower formation control of multiple UAVs with trajectory tracking design［J］. Acta Aeronautica et Astronautica Sinica， 2020， 41（S1）： 723758 （in Chinese）.
8	王祥科，陈浩，赵述龙. 大规模固定翼无人机集群编队控制方法［J］. 控制与决策， 2021， 36（9）： 2063-2073.
	WANG X K， CHEN H， ZHAO S L. Formation control of large-scale fixed-wing unmanned aerial vehicle swarms［J］. Control and Decision， 2021， 36（9）： 2063-2073 （in Chinese）.
9	ROSA V S M， BELO E M. Virtual structure formation flight control based on nonlinear MPC［C］∥2021 International Conference on Unmanned Aircraft Systems （ICUAS）. Piscataway： IEEE Press， 2021： 1383-1390.
10	SHEN Y K， WEI C. Multi-UAV flocking control with individual properties inspired by bird behavior［J］. Aerospace Science and Technology， 2022， 130： 107882.
11	姜龙亭，魏瑞轩，张启瑞，等. 基于群智机理的集群防碰撞控制［J］. 航空学报， 2020， 41（S2）： 724294.
	JIANG L T， WEI R X， ZHANG Q R， et al. Anti-collision control of UAVs based on swarm intelligence mechanism［J］. Acta Aeronautica et Astronautica Sinica， 2020， 41（S2）： 724294 （in Chinese）.
12	张令，段海滨，雍婷，等. 基于寒鸦配对交互行为的无人机集群编队控制［J］. 北京航空航天大学学报， 2021， 47（2）： 391-397.
	ZHANG L， DUAN H B， YONG T， et al. Unmanned aerial vehicle swarm formation control based on paired interaction mechanism in jackdaws［J］. Journal of Beijing University of Aeronautics and Astronautics， 2021， 47（2）： 391-397 （in Chinese）.
13	徐博，张大龙. 基于量子行为鸽群优化的无人机紧密编队控制［J］. 航空学报， 2020， 41（8）： 323722.
	XU B， ZHANG D L. Tight formation flight control of UAVs based on pigeon inspired algorithm optimization by quantum behavior［J］. Acta Aeronautica et Astronautica Sinica， 2020， 41（8）： 323722 （in Chinese）.
14	SUN D W， KWON C， HWANG I. Hybrid flocking control algorithm for fixed-wing aircraft［J］. Journal of Guidance， Control， and Dynamics， 2019， 42（11）： 2443-2455.
15	VÁSÁRHELYI G， VIRÁGH C， SOMORJAI G， et al. Optimized flocking of autonomous drones in confined environments［J］. Science Robotics， 2018， 3（20）： eaat3536.
16	WANG X， WANG S， LIANG X X， et al. Deep reinforcement learning： A survey［J］. IEEE Transactions on Neural Networks and Learning Systems， 2022， PP（99）： 1-15.
17	SILVER D， SCHRITTWIESER J， SIMONYAN K， et al. Mastering the game of Go without human knowledge［J］. Nature， 2017， 550（7676）： 354-359.
18	HUNG S M， GIVIGI S N. A Q-learning approach to flocking with UAVs in a stochastic environment［J］. IEEE Transactions on Cybernetics， 2016， 47（1）： 186-197.
19	WANG C， WANG J， ZHANG X D. A deep reinforcement learning approach to flocking and navigation of uavs in large-scale complex environments［C］∥2018 IEEE Global Conference on Signal and Information Processing （GlobalSIP）. Piscataway： IEEE Press， 2019： 1228-1232.
20	WANG C， YAN C， XIANG X J， et al. A continuous actor-critic reinforcement learning approach to flocking with fixed-wing UAVs［C］∥Asian Conference on Machine Learning. Berlin： Springer， 2019： 64-79.
21	SUTTON R S， BARTO A G. Reinforcement learning： An introduction［M］. 2nd ed. Cambridge： The MIT Press， 2017.
22	HAARNOJA T， ZHOU A， ABBEEL P， et al. Soft actor-critic： off-policy maximum entropy deep reinforcement learning with a stochastic actor［DB/OL］. arXiv preprint： 1801.01290， 2018.
23	García J， FERNÁNDEZ F. A comprehensive survey on safe reinforcement learning［J］. Journal of Machine Learning Research， 2015， 16： 1437-1480.
24	BRUNKE L， GREEFF M， HALL A W， et al. Safe learning in robotics： From learning-based control to safe reinforcement learning［J］. Annual Review of Control， Robotics， and Autonomous Systems， 2022， 5： 411-444.

[1]	Honglin ZHANG, Jianjun LUO, Weihua MA. Spacecraft game decision making for threat avoidance of space targets based on machine learning [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(8): 329136-329136.
[2]	Chuanyun WANG, Yang SU, Linlin WANG, Tian WANG, Jingjing WANG, Qian GAO. Multi-object continuous robust tracking algorithm for anti-UAV swarm [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(7): 329017-329017.
[3]	Shengzhe SHAN, Weiwei ZHANG. Air combat intelligent decision-making method based on self-play and deep reinforcement learning [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(4): 328723-328723.
[4]	Xuejian WANG, Yongming WEN, Xiaorong SHI, Ningning ZHANG, Jiexi LIU. Design of hybrid intelligent decision framework for multi⁃agent and multi⁃coupling tasks [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(S2): 729770-729770.
[5]	Xizhen GAO, Liang TANG, Huang HUANG. Deep reinforcement learning in autonomous manipulation for celestial bodies exploration: Applications and challenges [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2023, 44(6): 26762-026762.
[6]	Pan ZHOU, Jiangtao HUANG, Sheng ZHANG, Gang LIU, Bowen SHU, Jigang TANG. Intelligent air combat decision making and simulation based on deep reinforcement learning [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2023, 44(4): 126731-126731.
[7]	Xiangwei ZHU, Dan SHEN, Kai XIAO, Yuexin MA, Xiang LIAO, Fuqiang GU, Fangwen YU, Kefu GAO, Jingnan LIU. Mechanisms, algorithms, implementation and perspectives of brain⁃inspired navigation [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(19): 28569-028569.
[8]	Lei DONG, Hongbing CHEN, Xi CHEN, Changxiao ZHAO. Distributed multi-agent coalition task allocation strategy for single pilot operation mode based on DQN [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(13): 327895-327895.
[9]	Wenxue CHEN, Changsheng GAO, Wuxing JING. Trust region policy optimization guidance algorithm for intercepting maneuvering target [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2023, 44(11): 327596-327596.
[10]	Chao WEN, Wenhan DONG, XIE Wujie, Ming CAI, Ri LIU. Distributed cooperative area search method for UAV swarms based on revisit mechanism [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2023, 44(11): 327561-327561.
[11]	Sheng ZHANG, Pan ZHOU, Yang HE, Jiangtao HUANG, Gang LIU, Jigang TANG, Huaizhi JIA, Xin DU. Air combat maneuver decision-making test based on deep reinforcement learning [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2023, 44(10): 128094-128094.
[12]	SU Lingfei, HUA Yongzhao, DONG Xiwang, REN Zhang. Human-UAV swarm multi-modal intelligent interaction methods [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(S1): 727001-727001.
[13]	WANG Xiaoyue, WANG Xun, WANG Yongzhen, FEI Teng, LIU Dawei. Evaluation method for combat effectiveness of task-based simulated UAV swarm system [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(S1): 726937-726937.
[14]	FU Xiaowei, WANG Hui, XU Zhe. Cooperative pursuit strategy for multi-UAVs based on DE-MADDPG algorithm [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(5): 325311-325311.
[15]	XU Guangtong, WANG Zhu, CAO Yan, SUN Jingliang, LONG Teng. Dynamic-priority-decoupled UAV swarm trajectory planning using distributed sequential convex programming [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(2): 325059-325059.

Intelligent collaborative control of UAV swarms with collision avoidance safety constraints

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 24

Related Articles 15

Recommended Articles

Metrics

Comments