无人机状态检测Kalman滤波空地目标跟踪算法
收稿日期: 2023-11-06
修回日期: 2023-12-06
录用日期: 2024-02-28
网络出版日期: 2024-03-11
基金资助
国家自然科学基金(51979275);国家重点研发计划(2022YFD2001405);浙江省农业智能装备与机器人重点实验室开放课题(2023ZJZD2306);自然资源部超大城市自然资源时空大数据分析应用重点实验室开放基金(KFKT-2022-05);深圳市科技计划项目(ZDSYS20210623091808026);虚拟现实技术与系统全国重点实验室（北京航空航天大学）开放课题基金(VRLAB2022C10);能源清洁利用国家重点实验室开放基金课题(ZJUCEU2022002);农业农村部长三角智慧农业技术重点实验室开放基金(KSAT-YRD2023005);农业农村部华南热带智慧农业技术重点实验室开放课题(HNZHNY-KFKT-202202);高等教育科学研究规划课题重点课题(23XXK0304);中国农业大学2115人才工程项目
UAV object tracking for air⁃ground targets based on status detection and Kalman filter
Received date: 2023-11-06
Revised date: 2023-12-06
Accepted date: 2024-02-28
Online published: 2024-03-11
Supported by
National Natural Science Foundation of China(51979275);National Key Research and Development Program of China(2022YFD2001405);Open Fund of Key Laboratory of Intelligent Equipment and Robotics for Agriculture of Zhejiang Province(2023ZJZD2306);Key Laboratory of Spatial-temporal Big Data Analysis and Application of Natural Resources in Megacities, Ministry of Natural Resources(KFKT-2022-05);Shenzhen Science and Technology Program(ZDSYS20210623091808026);Open Project Program of State Key Laboratory of Virtual Reality Technology and Systems, Beihang University(VRLAB2022C10);Open Fund Project of State Key Laboratory of Clean Energy Utilization(ZJUCEU2022002);Open Fund of Key Laboratory of Smart Agricultural Technology （Yangtze River Delta）, Ministry of Agriculture and Rural Affairs(KSAT-YRD2023005);Open Project Program of Key Laboratory of Smart Agricultural Technology in Tropical South China, Ministry of Agriculture and Rural Affairs(HNZHNY-KFKT-202202);Higher Education Scientific Research Planning Project, China Association of Higher Education(23XXK0304);2115 Talent Development Program of China Agricultural University
针对无人机（UAV）面向空地目标进行目标跟踪过程中，发生目标离开视野、目标被遮挡、存在相似目标干扰等问题导致追踪失败的情况，提出一种基于追踪状态检测和Kalman滤波的重定位更新机制，将其与孪生全卷积网络（siamfc）跟踪器结合。以空地目标为被跟踪对象，以无人机为跟踪空地目标的跟踪者，首先，基于双峰选择、平均峰值相关能量变化率、最高响应值变化率和峰值旁瓣比变化率的检测机制检测当前的追踪状态是否异常，判断siamfc的追踪结果是否满足作为观测值的要求。其次，Kalman滤波利用目标运动的先验信息对追踪进行预测更新，当追踪状态异常时能够及时校正调整目标跟踪结果。基于LaSOT数据集完成训练，在UAV123航空数据集和自制的以无人机为目标的数据集上进行实时目标跟踪测试和对比实验。实验结果表明：该算法在UAV123上的精确率和成功率分别为66.0%和47.4%（62 帧/s），在自制的以无人机为目标的数据集上的精确率和成功率分别为72.0%和58.6%（55 帧/s），满足目标跟踪的实时性要求，且跟踪结果优于多数跟踪器。该算法在无人机为跟踪者和被跟踪对象的情况下均能完成有效目标跟踪，应对目标离开视野、部分或全部遮挡和存在相似目标干扰等挑战性场景的能力有所增强，且算法具有良好的泛化能力。
徐心宇 , 陈建 . 无人机状态检测Kalman滤波空地目标跟踪算法[J]. 航空学报, 2024 , 45(16) : 329834 -329834 . DOI: 10.7527/S1000-6893.2024.29834
In the UAV’s target tracking process， tracking failure will occur when the target leaves the field of view or is occluded， and similar target interference exists. To overcome these problems， a relocation mechanism based on the tracking status detection and Kalman filter is proposed， which is combined with the fully convolutional Siamese network （siamfc） tracker. In this paper， an air-ground target is taken as the object to be tracked， and UAV is taken as the tracker to track the air-ground target. First， the detection mechanism based on the selection mechanism for double peaks， changing rate of average peak correlation energy， changing rate of maximum response value， and changing rate of peak to sidelobe ratio is used to detect whether the current tracking status is abnormal or not， and determine whether the tracking results of the siamfc satisfy the requirement of being an observed value. Secondly， the Kalman filter utilizes a priori information of the target motion to predict and update the tracking result. When the tracking status is abnormal， it can correct and adjust the tracking result in time. We use the LaSOT video dataset to train the network. Real-time object tracking tests and comparison experiments are conducted on UAV123 aerial dataset and the customized dataset with UAV as the target. The experimental results show that the tracker optimized by the mechanism has an accuracy and success rate of 66.0% and 47.4% （62 frame/s） respectively on UAV123， and 72.0% and 58.6% （55 frame/s） respectively on the customized dataset with UAV as the target. The tracking results meet the real-time requirements of UAV object tracking， and are better than the results of most trackers. The algorithm accomplishes effective target tracking with both UAVs as trackers and tracked objects， and has an enhanced ability to cope with challenging scenarios such as the target’s leaving the field of view， partial or total occlusion of the target， and similar target interference. Meanwhile， the algorithm has good generalization.
| 1 | 钟晓伟， 王志胜， 丛玉华. 基于轻量孪生网络的无人机目标跟踪算法［J］. 弹箭与制导学报， 2023， 43（5）： 25-33. |
| ZHONG X W， WANG Z S， CONG Y H. Unmanned aerial vehicle target tracking algorithm based on lightweight Siamese networks［J］. Journal of Projectiles， Rockets， Missiles and Guidance， 2023， 43（5）： 25-33 （in Chinese）. | |
| 2 | 刘芳， 杨雨妍， 王鑫. 基于特征融合和分块注意力的无人机跟踪算法［J/OL］. 北京航空航天大学学报.（2023-09-27）［2023-11-01］.. |
| LIU F， YANG Y Y， WANG X. UAV tracking algorithm based on feature fusion and block attention［J］. Journal of Beijing University of Aeronautics and Astronautics.（2023-09-27）［2023-11-01］. （in Chinese）. | |
| 3 | 薛镇涛， 陈建， 张自超， 等. 基于复杂地块凸划分优化的多无人机覆盖路径规划［J］. 航空学报， 2022， 43（12）： 325990. |
| XUE Z T， CHEN J， ZHANG Z C， et al. Multi-UAV coverage path planning based on optimization of convex division of complex plots［J］. Acta Aeronautica et Astronautica Sinica， 2022， 43（12）： 325990 （in Chinese）. | |
| 4 | ZHANG Z C， WANG S B， CHEN J， et al. A bionic dynamic path planning algorithm of the micro UAV based on the fusion of deep neural network optimization/filtering and hawk-eye vision［J］. IEEE Transactions on Systems， Man， and Cybernetics： Systems， 2023， 53（6）： 3728-3740. |
| 5 | JIA J L， LAI Z Y， QIAN Y R， et al. Aerial video trackers review［J］. Entropy， 2020， 22（12）： 1358. |
| 6 | COMANICIU D， MEER P. Mean shift： A robust approach toward feature space analysis［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2002， 24（5）： 603-619. |
| 7 | BOLME D S， BEVERIDGE J R， DRAPER B A， et al. Visual object tracking using adaptive correlation filters［C］∥ 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2010： 2544-2550. |
| 8 | HENRIQUES J F， CASEIRO R， MARTINS P， et al. Exploiting the circulant structure of tracking-by-detection with kernels［C］∥ Computer Vision – ECCV 2012. Berlin， Heidelberg： Springer， 2012： 702-715. |
| 9 | HENRIQUES J F， CASEIRO R， MARTINS P， et al. High-speed tracking with kernelized correlation filters［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2015， 37（3）： 583-596. |
| 10 | LI Y， ZHU J K. A scale adaptive kernel correlation filter tracker with feature integration［C］∥AGAPITO L， BRONSTEIN M M， ROTHER C. Computer Vision - ECCV 2014 Workshops. Cham： Springer， 2015： 254-265. |
| 11 | DANELLJAN M， H?GER G， SHAHBAZ KHAN F， et al. Accurate scale estimation for robust visual tracking［C］∥ Proceedings of the British Machine Vision Conference. Durham：BMVA Press 2014：1-11. |
| 12 | LECUN Y， BOTTOU L， BENGIO Y， et al. Gradient-based learning applied to document recognition［J］. Proceedings of the IEEE， 1998， 86（11）： 2278-2324. |
| 13 | KRIZHEVSKY A， SUTSKEVER I， HINTON G E. ImageNet classification with deep convolutional neural networks［J］. Communications of the ACM， 2017， 60（6）： 84-90. |
| 14 | SIMONYAN K， ZISSERMAN A. Very deep convolutional networks for large-scale image recognition［DB/OL］.arXiv preprint： 1409.1556， 2014. |
| 15 | HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition［C］∥ 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2016： 770-778. |
| 16 | TAO R， GAVVES E， SMEULDERS A W M. Siamese instance search for tracking［C］∥ 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2016： 1420-1429. |
| 17 | BERTINETTO L， VALMADRE J， HENRIQUES J F， et al. Fully-convolutional Siamese networks for object tracking［C］∥ HUA G， J’EGOU H. Computer Vision?ECCV 2016 Workshops. Cham： Springer， 2016： 850-865. |
| 18 | 马玉民， 钱育蓉， 周伟航， 等. 基于孪生网络的目标跟踪算法综述［J］. 计算机工程与科学， 2023， 45（9）： 1578-1592. |
| MA Y M， QIAN Y R， ZHOU W H， et al. A survey of target tracking algorithms based on Siamese network［J］. Computer Engineering & Science， 2023， 45（9）： 1578-1592 （in Chinese）. | |
| 19 | ONDRA?OVI? M， TARáBEK P. Siamese visual object tracking： A survey［J］. IEEE Access， 2021，9： 110149-110172. |
| 20 | HE A F， LUO C， TIAN X M， et al. A twofold Siamese network for real-time object tracking［C］∥ 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2018： 4834-4843. |
| 21 | LI B， YAN J J， WU W， et al. High performance visual tracking with Siamese region proposal network［C］∥ 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2018： 8971-8980. |
| 22 | ZHU Z， WANG Q， LI B， et al. Distractor-aware Siamese networks for visual object tracking［C］∥ FERRARI， V， HEBERT M， SMINCHISESCU C， et al. Computer Vision?ECCV 2018 . Cham：Springer，2018： 103-119. |
| 23 | LI B， WU W， WANG Q， et al. SiamRPN++： Evolution of Siamese visual tracking with very deep networks［C］∥ 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2019： 4277-4286. |
| 24 | ZHANG W， SONG K， RONG X W， et al. Coarse-to-fine UAV target tracking with deep reinforcement learning［J］. IEEE Transactions on Automation Science and Engineering， 2018， 16（4）： 1522-1530. . |
| 25 | AVOLA D， CINQUE L， DIKO A， et al. MS-Faster R-CNN： Multi-stream backbone for improved Faster R-CNN object detection and aerial tracking from UAV images［J］. Remote Sensing， 2021， 13（9）： 1670. |
| 26 | LI Y M， FU C H， HUANG Z Y， et al. Intermittent contextual learning for keyfilter-aware UAV object tracking using deep convolutional feature［J］. IEEE Transactions on Multimedia， 2020， 23： 810-822. |
| 27 | BIAN Z Y， XU T F， CHEN J J， et al. Auto-learning correlation-filter-based target state estimation for real-time UAV tracking［J］. Remote Sensing， 2022， 14（21）： 5299. |
| 28 | 刘芳， 孙亚楠. 基于自适应融合网络的无人机目标跟踪算法［J］. 航空学报， 2022， 43（7）： 325522. |
| LIU F， SUN Y N. UAV target tracking algorithm based on adaptive fusion network［J］. Acta Aeronautica et Astronautica Sinica， 2022， 43（7）： 325522 （in Chinese）. | |
| 29 | 刘贞报， 马博迪， 高红岗， 等. 基于形态自适应网络的无人机目标跟踪方法［J］. 航空学报， 2021， 42（4）： 524904. |
| LIU Z B， MA B D， GAO H G， et al. Adaptive morphological network based UAV target tracking algorithm［J］. Acta Aeronautica et Astronautica Sinica， 2021， 42（4）： 524904 （in Chinese）. | |
| 30 | ZHOU L J， ZHANG J L. Combined Kalman filter and multifeature fusion Siamese network for real-time visual tracking［J］. Sensors， 2019， 19（9）： 2201. |
| 31 | WU F， ZHANG J L， XU Z Y. Stably adaptive anti-occlusion Siamese region proposal network for real-time object tracking［J］. IEEE Access， 2020， 8： 161349-161360. |
| 32 | SUN L F， ZHANG J J， YANG Z， et al. A motion-aware Siamese framework for unmanned aerial vehicle tracking［J］. Drones， 2023， 7（3）： 153. |
| 33 | WANG M M， LIU Y， HUANG Z Y. Large margin object tracking with circulant feature maps［C］∥ 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2017： 4800-4808. |
| 34 | BOLME D S， BEVERIDGE J R， DRAPER B A， et al. Visual object tracking using adaptive correlation filters［C］∥ 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2010： 2544-2550. |
| 35 | MUELLER M， SMITH N， GHANEM B. A benchmark and simulator for UAV tracking［M］∥ Computer Vision?ECCV 2016. Cham： Springer， 2016： 445-461. |
| 36 | WU Y， LIM J， YANG M H. Object tracking benchmark［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2015， 37（9）： 1834-1848. |
| 37 | FAN H， LIN L T， YANG F， et al. LaSOT： A high-quality benchmark for large-scale single object tracking［C］∥ 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2019： 5369-5378. |
| 38 | DANELLJAN M， H?GER G， KHAN F S， et al. Learning spatially regularized correlation filters for visual tracking［C］∥ 2015 IEEE International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2015： 4310-4318. |
| 39 | MA C， YANG X K， ZHANG C Y， et al. Long-term correlation tracking［C］∥ 2015 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2015： 5388-5396. |
| 40 | BERTINETTO L， VALMADRE J， GOLODETZ S， et al. Staple： Complementary learners for real-time tracking［C］∥ 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2016： 1401-1409. |
/
| 〈 |
|
〉 |
CJA