ACTA AERONAUTICAET ASTRONAUTICA SINICA >
UAV object tracking for air⁃ground targets based on status detection and Kalman filter
Received date: 2023-11-06
Revised date: 2023-12-06
Accepted date: 2024-02-28
Online published: 2024-03-11
Supported by
National Natural Science Foundation of China(51979275);National Key Research and Development Program of China(2022YFD2001405);Open Fund of Key Laboratory of Intelligent Equipment and Robotics for Agriculture of Zhejiang Province(2023ZJZD2306);Key Laboratory of Spatial-temporal Big Data Analysis and Application of Natural Resources in Megacities, Ministry of Natural Resources(KFKT-2022-05);Shenzhen Science and Technology Program(ZDSYS20210623091808026);Open Project Program of State Key Laboratory of Virtual Reality Technology and Systems, Beihang University(VRLAB2022C10);Open Fund Project of State Key Laboratory of Clean Energy Utilization(ZJUCEU2022002);Open Fund of Key Laboratory of Smart Agricultural Technology （Yangtze River Delta）, Ministry of Agriculture and Rural Affairs(KSAT-YRD2023005);Open Project Program of Key Laboratory of Smart Agricultural Technology in Tropical South China, Ministry of Agriculture and Rural Affairs(HNZHNY-KFKT-202202);Higher Education Scientific Research Planning Project, China Association of Higher Education(23XXK0304);2115 Talent Development Program of China Agricultural University
In the UAV’s target tracking process， tracking failure will occur when the target leaves the field of view or is occluded， and similar target interference exists. To overcome these problems， a relocation mechanism based on the tracking status detection and Kalman filter is proposed， which is combined with the fully convolutional Siamese network （siamfc） tracker. In this paper， an air-ground target is taken as the object to be tracked， and UAV is taken as the tracker to track the air-ground target. First， the detection mechanism based on the selection mechanism for double peaks， changing rate of average peak correlation energy， changing rate of maximum response value， and changing rate of peak to sidelobe ratio is used to detect whether the current tracking status is abnormal or not， and determine whether the tracking results of the siamfc satisfy the requirement of being an observed value. Secondly， the Kalman filter utilizes a priori information of the target motion to predict and update the tracking result. When the tracking status is abnormal， it can correct and adjust the tracking result in time. We use the LaSOT video dataset to train the network. Real-time object tracking tests and comparison experiments are conducted on UAV123 aerial dataset and the customized dataset with UAV as the target. The experimental results show that the tracker optimized by the mechanism has an accuracy and success rate of 66.0% and 47.4% （62 frame/s） respectively on UAV123， and 72.0% and 58.6% （55 frame/s） respectively on the customized dataset with UAV as the target. The tracking results meet the real-time requirements of UAV object tracking， and are better than the results of most trackers. The algorithm accomplishes effective target tracking with both UAVs as trackers and tracked objects， and has an enhanced ability to cope with challenging scenarios such as the target’s leaving the field of view， partial or total occlusion of the target， and similar target interference. Meanwhile， the algorithm has good generalization.
Xinyu XU , Jian CHEN . UAV object tracking for air⁃ground targets based on status detection and Kalman filter[J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2024 , 45(16) : 329834 -329834 . DOI: 10.7527/S1000-6893.2024.29834
| 1 | 钟晓伟， 王志胜， 丛玉华. 基于轻量孪生网络的无人机目标跟踪算法［J］. 弹箭与制导学报， 2023， 43（5）： 25-33. |
| ZHONG X W， WANG Z S， CONG Y H. Unmanned aerial vehicle target tracking algorithm based on lightweight Siamese networks［J］. Journal of Projectiles， Rockets， Missiles and Guidance， 2023， 43（5）： 25-33 （in Chinese）. | |
| 2 | 刘芳， 杨雨妍， 王鑫. 基于特征融合和分块注意力的无人机跟踪算法［J/OL］. 北京航空航天大学学报.（2023-09-27）［2023-11-01］.. |
| LIU F， YANG Y Y， WANG X. UAV tracking algorithm based on feature fusion and block attention［J］. Journal of Beijing University of Aeronautics and Astronautics.（2023-09-27）［2023-11-01］. （in Chinese）. | |
| 3 | 薛镇涛， 陈建， 张自超， 等. 基于复杂地块凸划分优化的多无人机覆盖路径规划［J］. 航空学报， 2022， 43（12）： 325990. |
| XUE Z T， CHEN J， ZHANG Z C， et al. Multi-UAV coverage path planning based on optimization of convex division of complex plots［J］. Acta Aeronautica et Astronautica Sinica， 2022， 43（12）： 325990 （in Chinese）. | |
| 4 | ZHANG Z C， WANG S B， CHEN J， et al. A bionic dynamic path planning algorithm of the micro UAV based on the fusion of deep neural network optimization/filtering and hawk-eye vision［J］. IEEE Transactions on Systems， Man， and Cybernetics： Systems， 2023， 53（6）： 3728-3740. |
| 5 | JIA J L， LAI Z Y， QIAN Y R， et al. Aerial video trackers review［J］. Entropy， 2020， 22（12）： 1358. |
| 6 | COMANICIU D， MEER P. Mean shift： A robust approach toward feature space analysis［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2002， 24（5）： 603-619. |
| 7 | BOLME D S， BEVERIDGE J R， DRAPER B A， et al. Visual object tracking using adaptive correlation filters［C］∥ 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2010： 2544-2550. |
| 8 | HENRIQUES J F， CASEIRO R， MARTINS P， et al. Exploiting the circulant structure of tracking-by-detection with kernels［C］∥ Computer Vision – ECCV 2012. Berlin， Heidelberg： Springer， 2012： 702-715. |
| 9 | HENRIQUES J F， CASEIRO R， MARTINS P， et al. High-speed tracking with kernelized correlation filters［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2015， 37（3）： 583-596. |
| 10 | LI Y， ZHU J K. A scale adaptive kernel correlation filter tracker with feature integration［C］∥AGAPITO L， BRONSTEIN M M， ROTHER C. Computer Vision - ECCV 2014 Workshops. Cham： Springer， 2015： 254-265. |
| 11 | DANELLJAN M， H?GER G， SHAHBAZ KHAN F， et al. Accurate scale estimation for robust visual tracking［C］∥ Proceedings of the British Machine Vision Conference. Durham：BMVA Press 2014：1-11. |
| 12 | LECUN Y， BOTTOU L， BENGIO Y， et al. Gradient-based learning applied to document recognition［J］. Proceedings of the IEEE， 1998， 86（11）： 2278-2324. |
| 13 | KRIZHEVSKY A， SUTSKEVER I， HINTON G E. ImageNet classification with deep convolutional neural networks［J］. Communications of the ACM， 2017， 60（6）： 84-90. |
| 14 | SIMONYAN K， ZISSERMAN A. Very deep convolutional networks for large-scale image recognition［DB/OL］.arXiv preprint： 1409.1556， 2014. |
| 15 | HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition［C］∥ 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2016： 770-778. |
| 16 | TAO R， GAVVES E， SMEULDERS A W M. Siamese instance search for tracking［C］∥ 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2016： 1420-1429. |
| 17 | BERTINETTO L， VALMADRE J， HENRIQUES J F， et al. Fully-convolutional Siamese networks for object tracking［C］∥ HUA G， J’EGOU H. Computer Vision?ECCV 2016 Workshops. Cham： Springer， 2016： 850-865. |
| 18 | 马玉民， 钱育蓉， 周伟航， 等. 基于孪生网络的目标跟踪算法综述［J］. 计算机工程与科学， 2023， 45（9）： 1578-1592. |
| MA Y M， QIAN Y R， ZHOU W H， et al. A survey of target tracking algorithms based on Siamese network［J］. Computer Engineering & Science， 2023， 45（9）： 1578-1592 （in Chinese）. | |
| 19 | ONDRA?OVI? M， TARáBEK P. Siamese visual object tracking： A survey［J］. IEEE Access， 2021，9： 110149-110172. |
| 20 | HE A F， LUO C， TIAN X M， et al. A twofold Siamese network for real-time object tracking［C］∥ 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2018： 4834-4843. |
| 21 | LI B， YAN J J， WU W， et al. High performance visual tracking with Siamese region proposal network［C］∥ 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2018： 8971-8980. |
| 22 | ZHU Z， WANG Q， LI B， et al. Distractor-aware Siamese networks for visual object tracking［C］∥ FERRARI， V， HEBERT M， SMINCHISESCU C， et al. Computer Vision?ECCV 2018 . Cham：Springer，2018： 103-119. |
| 23 | LI B， WU W， WANG Q， et al. SiamRPN++： Evolution of Siamese visual tracking with very deep networks［C］∥ 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2019： 4277-4286. |
| 24 | ZHANG W， SONG K， RONG X W， et al. Coarse-to-fine UAV target tracking with deep reinforcement learning［J］. IEEE Transactions on Automation Science and Engineering， 2018， 16（4）： 1522-1530. . |
| 25 | AVOLA D， CINQUE L， DIKO A， et al. MS-Faster R-CNN： Multi-stream backbone for improved Faster R-CNN object detection and aerial tracking from UAV images［J］. Remote Sensing， 2021， 13（9）： 1670. |
| 26 | LI Y M， FU C H， HUANG Z Y， et al. Intermittent contextual learning for keyfilter-aware UAV object tracking using deep convolutional feature［J］. IEEE Transactions on Multimedia， 2020， 23： 810-822. |
| 27 | BIAN Z Y， XU T F， CHEN J J， et al. Auto-learning correlation-filter-based target state estimation for real-time UAV tracking［J］. Remote Sensing， 2022， 14（21）： 5299. |
| 28 | 刘芳， 孙亚楠. 基于自适应融合网络的无人机目标跟踪算法［J］. 航空学报， 2022， 43（7）： 325522. |
| LIU F， SUN Y N. UAV target tracking algorithm based on adaptive fusion network［J］. Acta Aeronautica et Astronautica Sinica， 2022， 43（7）： 325522 （in Chinese）. | |
| 29 | 刘贞报， 马博迪， 高红岗， 等. 基于形态自适应网络的无人机目标跟踪方法［J］. 航空学报， 2021， 42（4）： 524904. |
| LIU Z B， MA B D， GAO H G， et al. Adaptive morphological network based UAV target tracking algorithm［J］. Acta Aeronautica et Astronautica Sinica， 2021， 42（4）： 524904 （in Chinese）. | |
| 30 | ZHOU L J， ZHANG J L. Combined Kalman filter and multifeature fusion Siamese network for real-time visual tracking［J］. Sensors， 2019， 19（9）： 2201. |
| 31 | WU F， ZHANG J L， XU Z Y. Stably adaptive anti-occlusion Siamese region proposal network for real-time object tracking［J］. IEEE Access， 2020， 8： 161349-161360. |
| 32 | SUN L F， ZHANG J J， YANG Z， et al. A motion-aware Siamese framework for unmanned aerial vehicle tracking［J］. Drones， 2023， 7（3）： 153. |
| 33 | WANG M M， LIU Y， HUANG Z Y. Large margin object tracking with circulant feature maps［C］∥ 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2017： 4800-4808. |
| 34 | BOLME D S， BEVERIDGE J R， DRAPER B A， et al. Visual object tracking using adaptive correlation filters［C］∥ 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2010： 2544-2550. |
| 35 | MUELLER M， SMITH N， GHANEM B. A benchmark and simulator for UAV tracking［M］∥ Computer Vision?ECCV 2016. Cham： Springer， 2016： 445-461. |
| 36 | WU Y， LIM J， YANG M H. Object tracking benchmark［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2015， 37（9）： 1834-1848. |
| 37 | FAN H， LIN L T， YANG F， et al. LaSOT： A high-quality benchmark for large-scale single object tracking［C］∥ 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2019： 5369-5378. |
| 38 | DANELLJAN M， H?GER G， KHAN F S， et al. Learning spatially regularized correlation filters for visual tracking［C］∥ 2015 IEEE International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2015： 4310-4318. |
| 39 | MA C， YANG X K， ZHANG C Y， et al. Long-term correlation tracking［C］∥ 2015 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2015： 5388-5396. |
| 40 | BERTINETTO L， VALMADRE J， GOLODETZ S， et al. Staple： Complementary learners for real-time tracking［C］∥ 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2016： 1401-1409. |
/
| 〈 |
|
〉 |
CJA