ACTA AERONAUTICAET ASTRONAUTICA SINICA >
Vision-based air-to-air multi-UAVs tracking
Received date: 2023-07-28
Revised date: 2023-08-22
Accepted date: 2023-11-08
Online published: 2023-12-07
Supported by
National Natural Science Foundation of China(62206020);Major Science and Technology Innovation Project of Shandong Province(2019SDZY05)
Vision-based air-to-air multi-object tracking is a key technology for UAV situational awareness. Recent research is limited to single-UAV tracking and migration of general multi-object tracking algorithms. To address the problem of inaccurate air-to-air multi-UAVs tracking， a cascade multi-UAVs tracking algorithm based on block feature enhancement extraction and local geometric association is designed. The UAV image is processed in blocks according to the characteristics of the fuselage and the arm， and the UAV’s fine-grained morphological features are extracted. The property that the relative geometric relationship between the UAVs is virtually invariant in the continuous frames is also utilized to extract the local geometric vectors of the UAVs. Then， the cascade association algorithm is designed by synthesizing the above technical components to improve the multi-object tracking algorithm’s ability to retrieve UAV objects and the association success rate， so as to improve the tracking performance of the proposed algorithm. Experiments show that in the test set， the proposed algorithm improves ID F1 Score （IDF1） by 5.6% compared to the state-of-the-art multi-object tracking algorithms OC-SORT， and improves Multiple Object Tracking Accuracy （MOTA） by 2.7% compared to the ByteTrack， which also performs well in the general multi-object tracking. The optimal performance for air-to-air multi-UAVs tracking can be realized， and the components used by the proposed algorithm can also be applied to SORT， BYTE and other data association algorithms to jointly improve their performance.
Zhaochen CHU , Tao SONG , Ren JIN , Defu LIN . Vision-based air-to-air multi-UAVs tracking[J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2024 , 45(14) : 629379 -629379 . DOI: 10.7527/S1000-6893.2023.29379
| 1 | 陈琳， 刘允刚. 面向无人机的视觉目标跟踪算法：综述与展望［J］. 信息与控制， 2022， 51（1）： 23-40. |
| CHEN L， LIU Y G. UAV visual target tracking algorithms： Review and future prospect［J］. Information and Control， 2022， 51（1）： 23-40 （in Chinese）. | |
| 2 | 闫超， 涂良辉， 王聿豪，等. 无人机在我国民用领域应用综述［J］. 飞行力学， 2022， 40（3）： 1-6. |
| YAN C， TU L H， WANG Y H， et al. Application of unmanned aerial vehicle in civil field in China［J］. Flight Dynamics， 2022， 40（3）： 1-6 （in Chinese）. | |
| 3 | 李仕帆. 无人机在消防灭火救援工作中的运用实践探讨［J］. 中国设备工程， 2023（8）： 8-10. |
| LI S F. Practical discussion on the utilization of UAVs in fire fighting and rescue work［J］. China Plant Engineering， 2023（8）： 8-10 （in Chinese）. | |
| 4 | 李浩， 孙合敏， 李宏权， 等. 无人机集群蜂群作战综述及其预警探测应对策略［J］. 飞航导弹， 2018（11）： 46-51. |
| LI H， SUN H M， LI H Q， et al. Overview of UAV cluster swarm combat and its early warning detection response strategy［J］. Aerodynamic Missile Journal， 2018（11）： 46-51 （in Chinese）. | |
| 5 | 刘雷， 刘大卫， 王晓光，等. 无人机集群与反无人机集群发展现状及展望［J］. 航空学报， 2022， 43（S1）： 726908. |
| LIU L， LIU D W， WANG X G，et al. Development status and outlook of UAV clusters and anti-UAV clusters［J］. Acta Aeronautica et Astronautica Sinica， 2022， 43（S1）： 726908 （in Chinese）. | |
| 6 | WANG C Y， SU Y， WANG J J， et al. UAVSwarm dataset： An unmanned aerial vehicle swarm dataset for multiple object tracking［J］. Remote Sensing， 2022， 14（11）： 2601. |
| 7 | 张洲宇， 曹云峰， 范彦铭. 低空小型无人机空域冲突视觉感知技术研究进展［J］. 航空学报， 2022， 43（8）： 025645. |
| ZHANG Z Y， CAO Y F， FAN Y M. Research progress of vision based aerospace conflict sensing technologies for small unmanned aerial vehicle in low altitude［J］. Acta Aeronautica et Astronautica Sinica， 2022， 43（8）： 025645 （in Chinese）. | |
| 8 | 薛远亮， 金国栋， 谭力宁， 等. 基于多尺度融合的自适应无人机目标跟踪算法［J］. 航空学报， 2023， 44（1）： 326107. |
| XUE Y L， JIN G D， TAN L N， et al. Adaptive UAV target tracking algorithm based on multi-scale fusion［J］. Acta Aeronautica et Astronautica Sinica， 2023， 44（1）： 326107 （in Chinese）. | |
| 9 | 刘芳， 孙亚楠. 基于自适应融合网络的无人机目标跟踪算法［J］. 航空学报， 2022， 43（7）： 325522. |
| LIU F， SUN Y N. UAV target tracking algorithm based on adaptive fusion network［J］. Acta Aeronautica et Astronautica Sinica， 2022， 43（7）： 325522 （in Chinese）. | |
| 10 | 刘贞报， 马博迪， 高红岗， 等. 基于形态自适应网络的无人机目标跟踪方法［J］. 航空学报， 2021， 42（4）： 524904. |
| LIU Z B， MA B D， GAO H G， et al. Adaptive morphological network based UAV target tracking algorithm［J］. Acta Aeronautica et Astronautica Sinica， 2021， 42（4）： 524904 （in Chinese）. | |
| 11 | DU Y H， WAN J F， ZHAO Y Y， et al. GIAOTracker： A comprehensive framework for MCMOT with global information and optimizing strategies in VisDrone 2021［C］∥2021 IEEE/CVF International Conference on Computer Vision Workshops. Piscataway： IEEE Press， 2021： 2809-2819. |
| 12 | SHEN H Y， HUANG Z Y， WANG H C， et al. Adaptive update of UAV multi-target tracking based on Transformer［C］∥2023 IEEE 6th Information Technology， Networking， Electronic and Automation Control Conference. Piscataway： IEEE Press， 2023： 6-13. |
| 13 | WANG G A， GU R S， LIU Z Z， et al. Track without appearance： Learn box and tracklet embedding with local and global motion patterns for vehicle tracking［C］∥2021 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE Press， 2021： 9856-9866. |
| 14 | WU H， NIE J H， HE Z W， et al. One-shot multiple object tracking in UAV videos using task-specific fine-grained features［J］. Remote Sensing， 2022， 14（16）： 3853. |
| 15 | 罗茜， 赵睿， 庄慧珊， 等. YOLOv5与Deep-SORT联合优化的无人机多目标跟踪算法［J］. 信号处理， 2022， 38（12）： 2628-2638. |
| LUO X， ZHAO R， ZHUANG H S， et al. UAV multi-target tracking algorithm jointly optimized by YOLOv5 and Deep-SORT［J］. Journal of Signal Processing， 2022， 38（12）： 2628-2638 （in Chinese）. | |
| 16 | 刘芳， 浦昭辉， 张帅超. 基于注意力特征融合的无人机多目标跟踪算法［J］. 控制与决策， 2023， 38（2）： 345-353. |
| LIU F， PU Z H， ZHANG S C. UAV multi-target tracking algorithm based on attention feature fusion［J］. Control and Decision， 2023， 38（2）： 345-353 （in Chinese）. | |
| 17 | VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need［C］∥Proceedings of the 31st International Conference on Neural Information Processing Systems. New York： ACM， 2017： 6000-6010. |
| 18 | ZHANG Y F， WANG C Y， WANG X G， et al. FairMOT： On the fairness of detection and re-identification in multiple object tracking［J］. International Journal of Computer Vision， 2021， 129（11）： 3069-3087. |
| 19 | ZHAO J， ZHANG J S， LI D D， et al. Vision-based anti-UAV detection and tracking［J］. IEEE Transactions on Intelligent Transportation Systems， 2022， 23（12）： 25323-25334. |
| 20 | CAO Z A， HUANG Z Y， PAN L， et al. TCTrack： Temporal contexts for aerial tracking［C］∥2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2022： 14778-14788. |
| 21 | LI Y M， FU C H， DING F Q， et al. AutoTrack： Towards high-performance visual tracking for UAV with automatic spatio-temporal regularization［C］∥2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2020： 11920-11929. |
| 22 | JIANG X L， LI P Z， LI Y J， et al. Graph neural based end-to-end data association framework for online multiple-object tracking［DB/OL］. arXiv preprint： 1907.05315， 2019. |
| 23 | ZHANG Y F， SUN P Z， JIANG Y， et al. ByteTrack： Multi-object tracking by associating every detection box［C］∥European Conference on Computer Vision. Cham： Springer， 2022： 1-21. |
| 24 | LUO W H， XING J L， MILAN A， et al. Multiple object tracking： A literature review［J］. Artificial Intelligence， 2021， 293： 103448. |
| 25 | WOJKE N， BEWLEY A， PAULUS D. Simple online and realtime tracking with a deep association metric［C］∥2017 IEEE International Conference on Image Processing. Piscataway： IEEE Press， 2017： 3645-3649. |
| 26 | DOSOVITSKIY A， BEYER L， KOLESNIKOV A， et al. An image is worth 16x16 words： Transformers for image recognition at scale［DB/OL］. arXiv preprint： 2010.11929， 2020. |
| 27 | REN S Q， HE K M， GIRSHICK R， et al. Faster R-CNN： Towards real-time object detection with region proposal networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（6）： 1137-1149. |
| 28 | BERGMANN P， MEINHARDT T， LEAL-TAIXé L. Tracking without bells and whistles［C］∥2019 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE Press， 2019： 941-951. |
| 29 | BEWLEY A， GE Z Y， OTT L， et al. Simple online and realtime tracking［C］∥2016 IEEE International Conference on Image Processing. Piscataway： IEEE Press， 2016： 3464-3468. |
| 30 | CAO J K， PANG J M， WENG X S， et al. Observation-centric SORT： Rethinking SORT for robust multi-object tracking［C］∥2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2023： 9686-9696. |
| 31 | ZHOU X Y， KOLTUN V， KR?HENBüHL P. Tracking objects as points［C］∥European Conference on Computer Vision. Cham： Springer， 2020： 474-490. |
| 32 | PENG J L， WANG C G， WAN F B， et al. Chained-tracker： Chaining paired attentive regression results for end-to-end joint multiple-object detection and tracking［C］∥European Conference on Computer Vision. Cham： Springer， 2020： 145-161. |
| 33 | GE Z， LIU S T， WANG F， et al. YOLOX： Exceeding YOLO series in 2021［DB/OL］. arXiv preprint： 2107.08430， 2021. |
| 34 | BOCHKOVSKIY A， WANG C Y， LIAO H Y M. YOLOv4： Optimal speed and accuracy of object detection［DB/OL］. arXiv preprint： 2004.10934， 2020. |
| 35 | ZHANG H Y， CISSE M， DAUPHIN Y N， et al. Mixup： Beyond empirical risk minimization［DB/OL］. arXiv preprint： 1710.09412， 2017. |
| 36 | GE Z， LIU S T， LI Z M， et al. OTA： Optimal transport assignment for object detection［C］∥2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2021： 303-312. |
| 37 | WANG C Y， MARK LIAO H Y， WU Y H， et al. CSPNet： A new backbone that can enhance learning capability of CNN［C］∥2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Piscataway： IEEE Press， 2020： 1571-1580. |
| 38 | LIU S， QI L， QIN H F， et al. Path aggregation network for instance segmentation［C］∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2018： 8759-8768. |
| 39 | REZATOFIGHI H， TSOI N， GWAK J Y， et al. Generalized intersection over union： A metric and a loss for bounding box regression［C］∥2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2019： 658-666. |
/
| 〈 |
|
〉 |
CJA