结合核相关滤波器和深度学习的运动相机中无人机目标检测

doi:10.7527/S1000-6893.2020.23733

ACTA AERONAUTICAET ASTRONAUTICA SINICA ›› 2020, Vol. 41 ›› Issue (9): 323733-323733.doi: 10.7527/S1000-6893.2020.23733

• Electronics and Electrical Engineering and Control • Previous Articles Next Articles

UAV detection in motion cameras combining kernelized correlation filters and deep learning

LIANG Dong^1,2,3, GAO Sai^1,2,3, SUN Han^1,2,3, LIU Ningzhong^1,2,3

1. College of Computer science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China;
2. MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, Nanjing 211106, China;
3. Collaborative Innovation Center of Novel Software Technology and Industrialization, Nanjing 211106, China

Received:2019-12-17 Revised:2020-01-07 Online:2020-09-15 Published:2020-03-06
Supported by:
National Natural Science Foundation of China (61601223); National Defense Technology Innovation Zone Project

Abstract

Abstract: To solve the problem of motion blurs caused by the rapid relative movement of the UAV and the camera, and the missed detection and false detection problems resulted from the lack of appearance information and complex background of small drones, a new drone detection-tracking method is proposed. Aiming at UAV targets with imaging sizes less than 32 pixel×32 pixel, an improved multi-layer feature pyramid classification and a target box regressor are proposed as target detectors to overcome missed detection. The detection result is used to initialize the target tracker based on kernelized correlation filters, and continuously modify the tracking result which provides a basis for the elimination of false detection. During the tracking process, a camera motion compensation strategy adaptive to the observed scene texture is introduced to achieve target relocation. Experimental results in multiple scenarios show that the proposed method is significantly better than traditional ones in the detection and tracking of small high-speed moving targets. In addition, the introduction of motion compensation mechanism further enhances the robustness of the method in extremely complex scenarios.

Key words: multi-layer feature pyramid, kernelized correlation filters, phase correlation, random sample consensus, optical flow

CLC Number:

V249
TP391

LIANG Dong, GAO Sai, SUN Han, LIU Ningzhong. UAV detection in motion cameras combining kernelized correlation filters and deep learning[J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2020, 41(9): 323733-323733.

References 26

[1]	GIRSHICK R. Fast R-CNN[C]//2015 IEEE International Conference on Computer Vision (ICCV). Piscataway:IEEE Press, 2016.
[2]	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once:Unified, real-time object detection[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2016.
[3]	LIU W, ANGUELOV D, ERHAN D, et al. SSD:Single shot MultiBox detector[C]//European Conference on Computer Vision. Berlin:Springer, 2016.
[4]	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 99:2999-3007.
[5]	HENRIQUES J F, CASEIRO R, MARTINS P, et al. High-speed tracking with kernelized correlation filters[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(3):583-596.
[6]	ROZANTSEV A, LEPETIT V, FUA P. Detecting flying objects using a single moving camera[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(5):879-892.
[7]	ROZANTSEV A, LEPETIT V, FUA P. Flying objects detection from a single moving camera[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2015.
[8]	LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Computer Society, 2017.
[9]	SZNITMAN R, BECKER C, FLEURET F, et al. Fast object detection with entropy-driven evaluation[C]//2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway:IEEE Press, 2013.
[10]	DOLLAR P, RABAUD V, COTTRELL G, et al. Behavior recognition via sparse spatio-temporal features[C]//2nd Joint IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance. Piscataway:IEEE Press, 2005.
[11]	刘芳, 杨安喆, 吴志威. 基于自适应Siamese网络的无人机目标跟踪算法[J]. 航空学报, 2020, 41(1):323423. LIU F, YANG A Z, WU Z W. UAV target tracking algorithm based on adaptive Siamese network[J]. Acta Aeronautica et Astronautica Sinica, 2020, 41(1):323423(in Chinese).
[12]	刘芳, 王洪娟, 黄光伟, 等. 基于自适应深度网络的无人机目标跟踪算法[J]. 航空学报, 2019, 40(3):322332. LIU F,WANG H J,HUANG G W, et al. UAV target tracking algorithm based on adaptive deep network[J]. Acta Aeronautica et Astronautica Sinica, 2019, 40(3):322332(in Chinese).
[13]	REDMON J, FARHADI A. YOLO9000:Better, faster, stronger[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway:IEEE Press, 2017:6517-6525.
[14]	REDMON J, FARHADI A. YOLOv3:An incremental improvement[C]//2018 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway:IEEE Press, 2018:13674-13682.
[15]	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway:IEEE Press, 2016:770-778.
[16]	LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 39(4):640-651.
[17]	KENNARD H R W. Ridge regression:Applications to nonorthogonal problems[J]. Technometrics, 1970, 12(1):69-82.
[18]	FISCHLER M A, BOLLES R C. Random sample consensus:A paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM, 1981, 24(6):381-395.
[19]	FOROOSH H, ZERUBIA J, BERTHOD M, et al. Extension of phase correlation to subpixel registration[J]. IEEE Transactions on Image Processing, 2002, 11(3):188-200.
[20]	ERTURK S. Digital image stabilization with sub-image phase correlation based global motion estimation[J]. IEEE Transactions on Consumer Electronics, 2003, 49(4):1320-1325.
[21]	LUCENA M, FUERTES J M, GOMEZ J I, et al. Optical flow-based probabilistic tracking[C]//Information Sciences Signal Processing and Their Applications, 2003:219-222.
[22]	SARKAR N, CHAUDHURI B B. An efficient differential box-counting approach to compute fractal dimension of image[J]. IEEE Transactions on Systems, Man and Cybernetics, 1994, 24(1):115-120.
[23]	ROLPH S. Fractal geometry:Mathematical foundations and applications[J]. Mathematical Gazette, 1990, 74(469):288-317.
[24]	LI B, YAN J, WU W, et al. High performance visual tracking with siamese region proposal network[C]//2018 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway:IEEE Press, 2018:8971-8980.
[25]	DANELLJAN M, BHAT G, KHAN F S, et al. ECO:Efficient convolution operators for tracking[EB/OL]. Computer Vision and Pattern Recognition(CVPR) (2017-04-10)[2019-12-15]. https//arxiv.org/abs/1611.09224.
[26]	BHAT G, DANELLJAN M, VAN GOOL L, et al. Learning discriminative model prediction for tracking[EB/OL]. Computer Vision and Pattern Recognition(CVPR) (2019-04-15)[2019-12-15]. https//arxiv.org/pdf/1904.07220.

UAV detection in motion cameras combining kernelized correlation filters and deep learning

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

References 26

Related Articles 3

Recommended Articles

Metrics

Comments

[1]	WANG Cong, WANG Haipeng, XIONG Wei, HE You. Refined track initiation algorithm for partly resolvable group targets based on phase correlation [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2017, 38(4): 320299-320299.
[2]	XU Dong;AN Jin-wen. Global Motion Estimation by Fitting Optical Flow in Aerial Video Imagery [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2006, 27(1): 94-97.
[3]	CHEN Zhen;GAO Man-tun;Yang Sheng-yun;SHEN Yun-wen. Calculation of 3D Motion and Structure from Optical Flow in Image Sequence [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2004, 25(1): 55-58.