基于多尺度融合的自适应无人机目标跟踪算法

doi:10.7527/S1000-6893.2021.26107

Abstract

Abstract:

To overcome the problems of small size， large variation of scale and interference of similar objects in the Unmanned Aerial Vehicle （UAV） tracking process， an adaptive UAV aerial target tracking algorithm is proposed based on multi-scale attention and feature fusion. Firstly， considering the abundant interference information in the UAV view， a deep and diverse feature extraction network is constructed to provide robust characterization of semantic features and diverse features of the target. Secondly， a multi-scale attention module is designed to suppress interference information while retaining target information at different scales. Then， the feature fusion module is used to fuse different layers of features to effectively integrate detailed and semantic information. Finally， multiple region proposal modules based on anchor-free strategy are used to adaptively perceive the scale variation of the target and make full use of the integrated feature information to achieve accurate localization and robust tracking of the target. The experiments show that the success and precision of the algorithm on the dataset are 61.7% and 81.5% with a speed of 40.5 frame/s. The algorithm has significantly enhanced target discrimination， scale perception and anti-interference capability， and can effectively cope with common challenges in the UAV tracking process.

Key words: unmanned aerial vehicle, object tracking, multi-scale, feature fusion, anchor free

CLC Number:

Yuanliang XUE, Guodong JIN, Lining TAN, Jiankun XU. Adaptive UAV target tracking algorithm based on multi-scale fusion[J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2023, 44(1): 326107.

Figures/Tables 18

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Fig. 9

Fig. 10

Table 1

Table 2

Fig. 11

Table 3

Fig. 12

Fig. 13

Fig. 14

Fig. 15

References 38

1	孟琭，杨旭. 目标跟踪算法综述［J］. 自动化学报， 2019， 45（7）： 1244-1260.
	MENG L， YANG X. A survey of object tracking algorithms［J］. Acta Automatica Sinica， 2019， 45（7）： 1244-1260 （in Chinese）.
2	TAO R， GAVVES E， SMEULDERS A W M. Siamese instance search for tracking［C］∥2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press，2016：1420-1429.
3	BERTINETTO L， VALMADRE J， HENRIQUES J F， et al. Fully-convolutional Siamese networks for object tracking［M］∥Lecture Notes in Computer Science. Cham： Springer International Publishing， 2016： 850-865.
4	HE A F， LUO C， TIAN X M， et al. A twofold Siamese network for real-time object tracking［C］∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press，2018：4834-4843.
5	LI B， YAN J J， WU W， et al. High performance visual tracking with Siamese region proposal network［C］∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press，2018：8971-8980.
6	REN S Q， HE K M， GIRSHICK R， et al. Faster R-CNN： Towards real-time object detection with region proposal networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（6）： 1137-1149.
7	FAN H， LING H B. Siamese cascaded region proposal networks for real-time visual tracking［C］∥2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2019：7944-7953.
8	LI B， WU W， WANG Q， et al. SiamRPN： evolution of Siamese visual tracking with very deep networks［C］∥2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press，2019：4277-4286.
9	HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition［C］∥2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press，2016：770-778.
10	刘芳，孙亚楠，王洪娟，等. 基于残差学习的自适应无人机目标跟踪算法［J］. 北京航空航天大学学报， 2020， 46（10）： 1874-1882.
	LIU F， SUN Y N， WANG H J， et al. Adaptive UAV target tracking algorithm based on residual learning［J］. Journal of Beijing University of Aeronautics and Astronautics， 2020， 46（10）： 1874-1882 （in Chinese）.
11	刘贞报，马博迪，高红岗，等. 基于形态自适应网络的无人机目标跟踪方法［J］. 航空学报， 2021， 42（4）： 524904.
	LIU Z B， MA B D， GAO H G， et al. Adaptive morphological network based UAV target tracking algorithm［J］. Acta Aeronautica et Astronautica Sinica， 2021， 42（4）： 524904 （in Chinese）.
12	刘芳，杨安喆，吴志威. 基于自适应Siamese网络的无人机目标跟踪算法［J］. 航空学报， 2020， 41（1）： 323423.
	LIU F， YANG A Z， WU Z W. Adaptive Siamese network based UAV target tracking algorithm［J］. Acta Aeronautica et Astronautica Sinica， 2020， 41（1）： 323423 （in Chinese）.
13	LAROCHELLE H， HINTON G. Learning to combine foveal glimpses with a third-order Boltzmann machine［C］∥Proceedings of the 23rd International Conference on Neural Information Processing Systems-Volume 1. New York： ACM， 2010：1243-1251.
14	WANG X L， GIRSHICK R， GUPTA A， et al. Non-local neural networks［C］∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2018：7794-7803.
15	HU J， SHEN L， ALBANIE S， et al. Squeeze-and-excitation networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2020， 42（8）： 2011-2023.
16	Alex K， Ilya S， E H G. ImageNet classification with deep convolutional neural networks［C］∥Proceedings of the 2012 neural information processing systems（NIPS）. New York： Curran Associates Inc， 2012.
17	SIMONYAN K， ZISSERMAN A. Very deep convolutional networks for large-scale image recognition［DB/OL］. arXiv preprint： 1409.1556，2014.
18	XIE S N， GIRSHICK R， DOLLÁR P， et al. Aggregated residual transformations for deep neural networks［C］∥2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2017：5987-5995.
19	SZEGEDY C， LIU W， JIA Y Q， et al. Going deeper with convolutions［C］∥2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2015：1-9.
20	DAI Y M， GIESEKE F， OEHMCKE S， et al. Attentional feature fusion［C］∥2021 IEEE Winter Conference on Applications of Computer Vision. Piscataway： IEEE Press， 2021：3559-3568.
21	YU Y C， XIONG Y L， HUANG W L， et al. Deformable Siamese attention networks for visual object tracking［C］∥2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2020：6727-6736.
22	柏罗，张宏立，王聪. 基于高效注意力和上下文感知的目标跟踪算法［J］. 北京航空航天大学学报， 2022， 48（7）： 1222-1232.
	BAI L， ZHANG H L， WANG C. Target tracking algorithm based on efficient attention and context awareness［J］. Journal of Beijing University of Aeronautics and Astronautics， 2022， 48（7）： 1222-1232 （in Chinese）.
23	PFLUGFELDER R. An In-depth analysis of visual tracking with Siamese neural networks［DB/OL］. arXiv preprint： 1707.00569， 2018.
24	LIN T Y， DOLLÁR P， GIRSHICK R， et al. Feature pyramid networks for object detection［C］∥2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2017：936-944.
25	HUANG L H， ZHAO X， HUANG K Q. GOT-10k： A large high-diversity benchmark for generic object tracking in the wild［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2021， 43（5）： 1562-1577.
26	CHEN Z D， ZHONG B N， LI G R， et al. Siamese box adaptive network for visual tracking［C］∥2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2020：6667-6676.
27	MUELLER M， SMITH N， GHANEM B. A benchmark and simulator for UAV tracking［C］∥Computer Vision-ECCV 2016. Cham： Springer International Publishing， 2016：445-461.
28	RUSSAKOVSKY O， DENG J， SU H， et al. ImageNet large scale visual recognition challenge［J］. International Journal of Computer Vision， 2015， 115（3）： 211-252.
29	CHATTOPADHAY A， SARKAR A， HOWLADER P， et al. Grad-CAM： generalized gradient-based visual explanations for deep convolutional networks［C］∥2018 IEEE Winter Conference on Applications of Computer Vision. Piscataway： IEEE Press，2018：839-847.
30	REAL E， SHLENS J， MAZZOCCHI S， et al. YouTube-BoundingBoxes： A large high-precision human-annotated data set for object detection in video［C］∥2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2017：7464-7473.
31	LIN T Y， MAIRE M， BELONGIE S， et al. Microsoft COCO： Common objects in context［C］∥Computer Vision-ECCV 2014. Cham： Springer International Publishing， 2014：740-755.
32	DANELLJAN M， HÄGER G， SHAHBAZ KHAN F， et al. Accurate scale estimation for robust visual tracking［C］∥Proceedings of the British Machine Vision Conference 2014. British Machine Vision Association， 2014：1-11.
33	ZHANG J M， MA S G， SCLAROFF S. MEEM： Robust tracking via multiple experts using entropy minimization［M］. Computer Vision-ECCV 2014. Cham： Springer International Publishing， 2014：188-203.
34	LI Y， ZHU J K. A scale adaptive kernel correlation filter tracker with feature integration［C］∥Computer Vision-ECCV 2014 Workshops. Cham： Springer International Publishing， 2015：254-265.
35	DANELLJAN M， HÄGER G， KHAN F S， et al. Learning spatially regularized correlation filters for visual tracking［C］∥2015 IEEE International Conference on Computer Vision. Piscataway： IEEE Press， 2015：4310-4318.
36	HARE S， GOLODETZ S， SAFFARI A， et al. Struck： structured output tracking with kernels［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2016， 38（10）： 2096-2109.
37	ZHU Z， WANG Q， LI B， et al. Distractor-aware Siamese networks for visual object tracking［C］∥Computer Vision-ECCV 2018. Cham： Springer International Publishing， 2018：103-119.
38	WU Y， LIM J， YANG M H. Object tracking benchmark［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2015， 37（9）： 1834-1848.

AlexNet	DF-ResNet	MS-AM	AFFM	AF-RPN	成功率/%	提升效果/%
√					55.7
	√				58.6	+2.9
	√	√			60.9	+5.2
	√		√		60.5	+4.8
	√			√	60.3	+4.6
	√	√	√	√	61.7	+6.0

网络	参数量/M	Top-1准确率/%
ResNet-101	44.55	76.6
ResNet-50	25.56	75.4
ResNet-50+GA	27.73	76.8
ResNet-50+GA+LA	29.90	77.7

视频序列	长度/帧	主要挑战
car7	1 033	SV，ARC，FOC，POC，VC，SOB
uav8	301	SV，LR，BC，IV
car15	469	SV，ARC，LR，POC，SOB
car18	1 207	SV，ARC，FM， VC，CM
wakeboard6	1 165	SV，ARC，LR，FM，IV，VC，CM

[1]	Chunhui ZHAO, Anmeng LIU, Yang LYU, Quan PAN. A survey of resilient self-localization for UAV [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(8): 28839-028839.
[2]	Wei ZHANG, Ruojun HE. Autonomous trajectory design for IoT data collection by UAV [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(8): 329054-329054-1.
[3]	Chuanyun WANG, Yang SU, Linlin WANG, Tian WANG, Jingjing WANG, Qian GAO. Multi-object continuous robust tracking algorithm for anti-UAV swarm [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(7): 329017-329017.
[4]	Xudong LUO, Yiquan WU, Jinlin CHEN. Research progress on deep learning methods for object detection and semantic segmentation in UAV aerial images [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(6): 28822-028822.
[5]	Chao AN, Guixi HUO, Yang MENG, Changchuan XIE, Chao YANG. Aerodynamic modeling methods and influence of layout parameters for wingtip⁃hinged multi⁃body combined UAV [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(6): 629587-629587.
[6]	Gaojie ZHENG, Xiaoming HE, Dongpo LI, Huijun TAN, Kun WANG, Zhenlong WU, Depeng WANG. Double 90° deflection inlet/volute coupling flow characteristics of tail-powered unmanned aerial vehicle [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(4): 128782-128782.
[7]	Shuangxi LIU, Zehuai LIN, Wei LIU, Binbin YAN, Wei HUANG. Transition mode control scheme of tilt rotor UAV based on INDI [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(17): 529685-529685.
[8]	Qian ZHANG, Guanwei YAN, Qin NIE, Ruihai CHEN, Jianing LIU. Aircraft-missile cooperative guidance method based on trajectory numerical optimization of long-range air-to-air missiles [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(17): 530138-530138.
[9]	Chuangxin ZHAO, Haojian TI, Zhaohong LI, Shan FENG, Shuxing YANG, Peng TANG. Multi⁃UAV cooperative formation control and flight test verification [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(17): 530249-530249.
[10]	Xinyu XU, Jian CHEN. UAV object tracking for air⁃ground targets based on status detection and Kalman filter [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(16): 329834-329834.
[11]	Jiang ZHAO, Xuan ZHANG, Pei CHI, Yingxun WANG. Self⁃adaptive formation control and dynamic path planning for air⁃ground heterogeneous swarm [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(16): 329809-329809.
[12]	Wu PAN, Xiang WANG, Na LYU, Zhiyuan REN, Wenya LIU. Multi-UAV relay connectivity optimization with constraint of hover time [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(16): 329854-329854.
[13]	Honghong ZHANG, Wenhua LI, Jiayi ZHENG, Hongbin LIU, Peng ZHANG, Peng GAO, Xusheng GAN. Manned/unmanned aerial vehicle cooperative combat system: Concepts, technologies, and challenges [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(15): 29653-029653.
[14]	Yanfang LIU, Jiayu SHE, Qiufan YUAN, Rui ZHOU, Naiming QI. Real⁃time small target detection networks for UAV remote sensing [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(14): 630119-630119.
[15]	Xinlin XIAO, Weichao SHI, Xiangtao ZHENG, Yueming GAO, Xiaoqiang LU. Multiple models collaboration for ship detection [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(14): 630241-630241.

Adaptive UAV target tracking algorithm based on multi-scale fusion

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 18

References 38

Related Articles 15

Recommended Articles

Metrics

Comments