ACTA AERONAUTICAET ASTRONAUTICA SINICA >
Intelligent collaborative control of UAV swarms with collision avoidance safety constraints
Received date: 2023-10-08
Revised date: 2023-10-08
Accepted date: 2023-10-10
Online published: 2023-10-13
An intelligent cooperative control method for Unmanned Aerial Vehicle （UAV） swarms combining Deep Reinforcement Learning （DRL） and the collision avoidance strategy is proposed. First， the problem of UAV swarm control is described， and a motion model of the UAV is established. Secondly， under the framework of deep reinforcement learning， the training mechanism of the UAV swarm control strategy is established. By analyzing the observation space and action space of the UAV， the network structure of the UAV swarm control strategy is designed. The reward function is designed with the safety， tightness and consistency of the UAV swarm as the main goal. To solve the problem of lack of safety guarantee in the DRL method， the collision avoidance strategy of the UAV swarm is designed at the same time， and the collision avoidance commands can be calculated according to the relative motion states between UAVs and between UAVs and threat areas to avoid UAVs collision. The simulation results show that the proposed control method can effectively avoid the risk of collision between UAVs， guarantee the safety of the UAV swarm， and make the UAV swarm more compact.
Yunpeng CAI , Dapeng ZHOU , Jiangchuan DING . Intelligent collaborative control of UAV swarms with collision avoidance safety constraints[J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2024 , 45(5) : 529683 -529683 . DOI: 10.7527/S1000-6893.2023.29683
| 1 | 刘伯健， 李爱军， 郭永， 等. 带有输入受限的无人机精确编队合围容错控制［J］. 航空学报， 2023， 44（9）： 327414. |
| LIU B J， LI A J， GUO Y， et al. Fault-tolerant containment control for precise formation of UAVs with input saturation［J］. Acta Aeronautica et Astronautica Sinica， 2023， 44（9）： 327414 （in Chinese）. | |
| 2 | 刘树光， 刘荣华， 王欢， 等. 国外无人机集群协同控制技术新进展［J］. 飞航导弹， 2021（8）： 24-31. |
| LIU S G， LIU R H， WANG H， et al. New progress of cooperative control technology of UAV cluster abroad［J］. Aerodynamic Missile Journal， 2021（8）： 24-31 （in Chinese）. | |
| 3 | 吴杰宏， 李丹阳. 无人机集群编队控制方法研究综述［J］. 无线电通信技术， 2023， 49（4）： 589-596. |
| WU J H， LI D Y. A review of UAV cluster formation control methods［J］. Radio Communications Technology， 2023， 49（4）： 589-596 （in Chinese）. | |
| 4 | ALI Z A， ISRAR A， ALKHAMMASH E H， et al. A leader-follower formation control of multi-UAVs via an adaptive hybrid controller［J］. Complexity， 2021， 2021： 1-16. |
| 5 | CHEN Q Y， WANG Y J， LU Y F. Formation control for UAVs based on the virtual structure idea and nonlinear guidance logic［C］∥2021 6th International Conference on Automation， Control and Robotics Engineering （CACRE）. Piscataway： IEEE Press， 2021： 135-139. |
| 6 | 吴宇， 梁天骄. 基于改进一致性算法的无人机编队控制［J］. 航空学报， 2020， 41（9）： 323848. |
| WU Y， LIANG T J. Improved consensus-based algorithm for unmanned aerial vehicle formation control［J］. Acta Aeronautica et Astronautica Sinica， 2020， 41（9）： 323848 （in Chinese）. | |
| 7 | 王晶， 顾维博， 窦立亚. 基于Leader-Follower的多无人机编队轨迹跟踪设计［J］. 航空学报， 2020， 41（S1）： 723758. |
| WANG J， GU W B， DOU L Y. Leader-Follower formation control of multiple UAVs with trajectory tracking design［J］. Acta Aeronautica et Astronautica Sinica， 2020， 41（S1）： 723758 （in Chinese）. | |
| 8 | 王祥科， 陈浩， 赵述龙. 大规模固定翼无人机集群编队控制方法［J］. 控制与决策， 2021， 36（9）： 2063-2073. |
| WANG X K， CHEN H， ZHAO S L. Formation control of large-scale fixed-wing unmanned aerial vehicle swarms［J］. Control and Decision， 2021， 36（9）： 2063-2073 （in Chinese）. | |
| 9 | ROSA V S M， BELO E M. Virtual structure formation flight control based on nonlinear MPC［C］∥2021 International Conference on Unmanned Aircraft Systems （ICUAS）. Piscataway： IEEE Press， 2021： 1383-1390. |
| 10 | SHEN Y K， WEI C. Multi-UAV flocking control with individual properties inspired by bird behavior［J］. Aerospace Science and Technology， 2022， 130： 107882. |
| 11 | 姜龙亭， 魏瑞轩， 张启瑞， 等. 基于群智机理的集群防碰撞控制［J］. 航空学报， 2020， 41（S2）： 724294. |
| JIANG L T， WEI R X， ZHANG Q R， et al. Anti-collision control of UAVs based on swarm intelligence mechanism［J］. Acta Aeronautica et Astronautica Sinica， 2020， 41（S2）： 724294 （in Chinese）. | |
| 12 | 张令， 段海滨， 雍婷， 等. 基于寒鸦配对交互行为的无人机集群编队控制［J］. 北京航空航天大学学报， 2021， 47（2）： 391-397. |
| ZHANG L， DUAN H B， YONG T， et al. Unmanned aerial vehicle swarm formation control based on paired interaction mechanism in jackdaws［J］. Journal of Beijing University of Aeronautics and Astronautics， 2021， 47（2）： 391-397 （in Chinese）. | |
| 13 | 徐博， 张大龙. 基于量子行为鸽群优化的无人机紧密编队控制［J］. 航空学报， 2020， 41（8）： 323722. |
| XU B， ZHANG D L. Tight formation flight control of UAVs based on pigeon inspired algorithm optimization by quantum behavior［J］. Acta Aeronautica et Astronautica Sinica， 2020， 41（8）： 323722 （in Chinese）. | |
| 14 | SUN D W， KWON C， HWANG I. Hybrid flocking control algorithm for fixed-wing aircraft［J］. Journal of Guidance， Control， and Dynamics， 2019， 42（11）： 2443-2455. |
| 15 | VáSáRHELYI G， VIRáGH C， SOMORJAI G， et al. Optimized flocking of autonomous drones in confined environments［J］. Science Robotics， 2018， 3（20）： eaat3536. |
| 16 | WANG X， WANG S， LIANG X X， et al. Deep reinforcement learning： A survey［J］. IEEE Transactions on Neural Networks and Learning Systems， 2022， PP（99）： 1-15. |
| 17 | SILVER D， SCHRITTWIESER J， SIMONYAN K， et al. Mastering the game of Go without human knowledge［J］. Nature， 2017， 550（7676）： 354-359. |
| 18 | HUNG S M， GIVIGI S N. A Q-learning approach to flocking with UAVs in a stochastic environment［J］. IEEE Transactions on Cybernetics， 2016， 47（1）： 186-197. |
| 19 | WANG C， WANG J， ZHANG X D. A deep reinforcement learning approach to flocking and navigation of uavs in large-scale complex environments［C］∥2018 IEEE Global Conference on Signal and Information Processing （GlobalSIP）. Piscataway： IEEE Press， 2019： 1228-1232. |
| 20 | WANG C， YAN C， XIANG X J， et al. A continuous actor-critic reinforcement learning approach to flocking with fixed-wing UAVs［C］∥Asian Conference on Machine Learning. Berlin： Springer， 2019： 64-79. |
| 21 | SUTTON R S， BARTO A G. Reinforcement learning： An introduction［M］. 2nd ed. Cambridge： The MIT Press， 2017. |
| 22 | HAARNOJA T， ZHOU A， ABBEEL P， et al. Soft actor-critic： off-policy maximum entropy deep reinforcement learning with a stochastic actor［DB/OL］. arXiv preprint： 1801.01290， 2018. |
| 23 | García J， FERNáNDEZ F. A comprehensive survey on safe reinforcement learning［J］. Journal of Machine Learning Research， 2015， 16： 1437-1480. |
| 24 | BRUNKE L， GREEFF M， HALL A W， et al. Safe learning in robotics： From learning-based control to safe reinforcement learning［J］. Annual Review of Control， Robotics， and Autonomous Systems， 2022， 5： 411-444. |
/
| 〈 |
|
〉 |
CJA