无人机视觉引导对接过程中的协同目标检测

王辉; 贾自凯; 金忍; 林德福; 范军芳; 徐超

doi:10.7527/S1000-6893.2020.24854

航空学报 >

2022 , Vol. 43 >Issue 1: 324854 - 324854

DOI: https://doi.org/10.7527/S1000-6893.2020.24854

电子电气工程与控制

无人机视觉引导对接过程中的协同目标检测

王辉 ,
贾自凯 ,
金忍 ,
林德福 ,
范军芳 ,
徐超

展开

1. 北京理工大学宇航学院, 北京 100081;
2. 北京理工大学无人机自主控制技术北京市重点实验室, 北京 100081;
3. 北京信息科技大学高动态导航技术北京市重点实验室, 北京 100085;
4. 北京特种机电研究所, 北京 100012

收稿日期: 2020-10-12

修回日期: 2020-12-10

网络出版日期: 2020-12-03

基金资助

国家自然科学基金（U1613225）；北京信息科技大学高动态导航技术北京市重点实验室资助项目（HDN2020105）；北京信息科技大学高动态导航技术北京市重点实验室开放课题基金（HDN2020101）

收起

Cooperative object detection in UAV-based vision-guided docking

WANG Hui ,
JIA Zikai ,
JIN Ren ,
LIN Defu ,
FAN Junfang ,
XU Chao

Expand

1. School of Aerospace Engineering, Beijing Institute of Technology, Beijing 100081, China;
2. Beijing Key Laboratory of UAV Autonomous Control, Beijing Institute of Technology, Beijing 100081, China;
3. Beijing Key Laboratory of High-Dynamic Navigation Technology, Beijing Information Science and Technology University, Beijing 100085, China;
4. Beijing Institute of Special Mechanic-Electric, Beijing 100012, China

Received date: 2020-10-12

Revised date: 2020-12-10

Online published: 2020-12-03

Supported by

National Natural Science Foundation of China(U1613225); Funded Project of Beijing Key Laboratory of High Dynamic Navigation Technology of Beijing Information Science and Technology University(HDN2020105); Open Fund Project of Beijing Key Laboratory of High Dynamic Navigation Technology of Beijing Information Science and Technology University(HDN2020101)

Fold

摘要

无人机空中自主回收是未来的发展趋势，对空中载具的自动识别是实现视觉引导回收的关键技术之一。目前对于空中关联目标检测的研究局限于单个目标个体，没有充分利用关联目标之间的信息。本文针对空中高动态对接中的关联目标检测问题，提出了一种母机与挂载对接物体的单阶段快速协同目标检测算法，包括相关类别并行独立分支目标检测、相关类别掩模增强检测以及相关类别的特征一致性约束，这些模块能够共同提升检测表现。实验表明，在测试集中该算法相比于YOLOv4能够提升4.3%的平均精度，相比YOLOv3-Tiny能够提升31.6%的平均精度。同时，该算法已应用在MBZIRC2020的高动态空中对接项目上，实现机载图像在线实时处理，团队借此斩获冠军。

关键词： 无人机自主回收; 视觉引导; 目标检测; 关联目标; 卷积神经网络

本文引用格式

王辉 , 贾自凯 , 金忍 , 林德福 , 范军芳 , 徐超 . 无人机视觉引导对接过程中的协同目标检测[J]. 航空学报, 2022 , 43(1) : 324854 -324854 . DOI: 10.7527/S1000-6893.2020.24854

Abstract

Autonomous aerial recovery of UAV is a future development trend, and automatic detection of aerial vehicles is one of the key technologies to realize vision-guided recovery. At present, the research on the detection of aerial related objects is limited to individual objects, and the information between correlated objects is not fully utilized. For the problem of related object detection in high-dynamic aerial docking, this paper proposes a single-stage fast cooperative algorithm for detection of the master and the mount, including detection of sibling independent head of related category, detection of mask enhancement of related category, and constraints on consistency of features of related categories. These modules can improve the detection performance jointly. Experiments show that in the test dataset, the algorithm can obtain a 4.3% increase of the average precision of compared with YOLOv4, and can obtain a 31.6% increase of the average precision compared with YOLOv3-Tiny. At the same time, this algorithm has been applied to the high dynamic aerial docking project of MBZIRC2020 to achieve online real-time processing of airborne images, and our team won the championship.

Key words： UAV autonomous recovery; visual guided; object detection; related objects; convolutional neural networks

参考文献

[1] 邹立岩, 张明智, 荣明. 智能无人机集群概念及主要发展趋势分析[J]. 战术导弹技术, 2019(5): 1-11, 43. ZOU L Y, ZHANG M Z, RONG M. Analysis of intelligent unmanned aircraft systems swarm concept and main development trend[J]. Tactical Missile Technology, 2019(5): 1-11, 43(in Chinese).
[2] 贾永楠, 田似营, 李擎. 无人机集群研究进展综述[J]. 航空学报, 2020, 41(S1): 723738. JIA Y N, TIAN S Y, LI Q. Recent development of unmanned aerial vehicle swarms[J]. Acta Aeronautica et Astronautica Sinica, 2020, 41(S1): 723738(in Chinese).
[3] 王祥科, 刘志宏, 丛一睿, 等. 小型固定翼无人机集群综述和未来发展[J]. 航空学报, 2020, 41(4): 023732. WANG X K, LIU Z H, CONG Y R, et al. Miniature fixed-wing UAV swarms: Review and outlook[J]. Acta Aeronautica et Astronautica Sinica, 2020, 41(4): 023732(in Chinese).
[4] 牛轶峰, 肖湘江, 柯冠岩. 无人机集群作战概念及关键技术分析[J]. 国防科技, 2013, 34(5): 37-43. NIU Y F, XIAO X J, KE G Y. Operation concept and key techniques of unmanned aerial vehicle swarms[J]. National Defense Science & Technology, 2013, 34(5): 37-43(in Chinese).
[5] 李俊国. 蜂群式固定翼无人机空基回收系统设计及动力学分析[D]. 哈尔滨: 哈尔滨工业大学, 2017. LI J G. Design and dynamics analysis of air base recovery system for colony type fixed wing UAV[D]. Harbin: Harbin Institute of Technology, 2017(in Chinese).
[6] VENDRA S, CAMPA G, NAPOLITANO M R, et al. Addressing corner detection issues for machine vision based UAV aerial refueling[J]. Machine Vision and Applications, 2007, 18(5): 261-273.
[7] DELL’AQUILA R V, CAMPA G, NAPOLITANO M R, et al. Real-time machine-vision-based position sensing system for UAV aerial refueling[J]. Journal of Real-Time Image Processing, 2007, 1(3): 213-224.
[8] HARRIS C, STEPHENS M. A combined corner and edge detector[C]//Proceedings ofthe Alvey Vision Conference, 1988: 147-151.
[9] SMITH S M, BRADY J M. Susan-a new approach to low level image processing[J]. International Journal of Computer Vision, 1997, 23(1): 45-78.
[10] CAMPA G, MAMMARELLA M, NAPOLITANO M R, et al. A comparison of pose estimation algorithms for machine vision based aerial refueling for UAVs[C]//2006 14th Mediterranean Conference on Control and Automation. Piscataway: IEEE Press, 2006: 1-6.
[11] FRAVOLINI M L, CAMPA G, NAPOLITANO M R. Evaluation of machine vision algorithms for autonomous aerial refueling for unmanned aerial vehicles[J]. Journal of Aerospace Computing, Information, and Communication, 2007, 4(9): 968-985.
[12] MASH R, BORGHETTI B, PECARINA J. Improved aircraft recognition for aerial refueling through data augmentation in convolutional neural networks[C]//Advances in Visual Computing, 2016.
[13] WANG X F, KONG X W, ZHI J H, et al. Real-time drogue recognition and 3D locating for UAV autonomous aerial refueling based on monocular machine vision[J]. Chinese Journal of Aeronautics, 2015, 28(6): 1667-1675.
[14] YIN Y J, WANG X G, XU D, et al. Robust visual detection-learning-tracking framework for autonomous aerial refueling of UAVs[J]. IEEE Transactions on Instrumentation and Measurement, 2016, 65(3): 510-521.
[15] CHEN A G, WANG B X, YANG C T, et al. Research on drogue detection algorithm for aerial refueling(IEEE/CSAA GNCC)[C]//2018 IEEE CSAA Guidance, Navigation and Control Conference(CGNCC). Piscataway: IEEE Press, 2018: 1-4.
[16] TU Z W, BAI X. Auto-context and its application to high-level vision tasks and 3D brain image segmentation[C]//IEEE Transactions on Pattern Analysis and Machine Intelligence. Piscataway: IEEE Press, 2010: 1744-1757.
[17] GALLEGUILLOS C, RABINOVICH A, BELONGIE S. Object categorization using co-occurrence, location and appearance[C]//2008 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2008: 1-8.
[18] TORRALBA, MURPHY, FREEMAN, et al. Context-based vision system for place and object recognition[C]//Proceedings Ninth IEEE International Conference on Computer Vision. Piscataway: IEEE Press, 2003: 273-280.
[19] FELZENSZWALB P F, GIRSHICK R B, MCALLESTER D, et al. Object detection with discriminatively trained part-based models[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(9): 1627-1645.
[20] BIEDERMAN I, MEZZANOTTE R J, RABINOWITZ J C. Scene perception: Detecting and judging objects undergoing relational violations[J]. Cognitive Psychology, 1982, 14(2): 143-177.
[21] CHOI M J, TORRALBA A, WILLSKY A S. A tree-based context model for object recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(2): 240-252.
[22] MOTTAGHI R, CHEN X J, LIU X B, et al. The role of context for object detection and semantic segmentation in the wild[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2014: 891-898.
[23] HU H, GU J Y, ZHANG Z, et al. Relation networks for object detection[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 3588-3597.
[24] 张咪, 赵勇, 布树辉, 等. 基于阶层标识的无人机自主精准降落系统[J]. 航空学报, 2018, 39(10): 322150. ZHANG M, ZHAO Y, BU S H, et al. Multi-level marker based autonomous landing system for UAVs[J]. Acta Aeronautica et Astronautica Sinica, 2018, 39(10): 322150(in Chinese).
[25] 高嘉瑜, 袁苏哲, 景鑫, 等. 基于AprilTag二维码的无人机着陆引导方法[J]. 现代导航, 2020, 11(1): 20-25. GAO J Y, YUAN S Z, JING X, et al. Method of UAV position based on cooperative two-dimension code[J]. Modern Navigation, 2020, 11(1): 20-25(in Chinese).
[26] 赵文一. 无人机视觉辅助自主降落系统研究[D]. 哈尔滨: 哈尔滨工业大学, 2018. ZHAO W Y. Research on vision-based autonomous landing system of UAV[D]. Harbin: Harbin Institute of Technology, 2018(in Chinese).
[27] FALANGA D, ZANCHETTIN A, SIMOVIC A, et al. Vision-based autonomous quadrotor landing on a moving platform[C]//2017 IEEE International Symposium on Safety, Security and Rescue Robotics(SSRR). Piscataway: IEEE Press, 2017: 200-207.
[28] BACA T, STEPAN P, SASKA M. Autonomous landing on a moving car with unmanned aerial vehicle[C]//2017 European Conference on Mobile Robots(ECMR). Piscataway: IEEE Press, 2017: 1-6.
[29] TRUONG N Q, NGUYEN P H, NAM S H, et al. Deep learning-based super-resolution reconstruction and marker detection for drone landing[J]. IEEE Access, 2019, 7: 61639-61655.
[30] SUN S Y, YIN Y J, WANG X G, et al. Robust visual detection and tracking strategies for autonomous aerial refueling of UAVs[J]. IEEE Transactions on Instrumentation and Measurement, 2019, 68(12): 4640-4652.
[31] 江波, 屈若锟, 李彦冬, 等. 基于深度学习的无人机航拍目标检测研究综述[J]. 航空学报, 2021, 42(4): 524519. JIANG B, QU R K, LI Y D, et al. Object detection in UAV imagery based on deep learning: Review[J]. Acta Aeronautica et Astronautica Sinica, 2021, 42(4): 524519(in Chinese).
[32] 方路平, 何杭江, 周国民. 目标检测算法研究综述[J]. 计算机工程与应用, 2018, 54(13): 11-18, 33. FANG L P, HE H J, ZHOU G M. Research overview of object detection methods[J]. Computer Engineering and Applications, 2018, 54(13): 11-18, 33(in Chinese).
[33] TIAN Z, SHEN C H, CHEN H, et al. FCOS: Fully convolutional one-stage object detection[C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV). Piscataway: IEEE Press, 2019: 9626-9635.
[34] HE K M, GKIOXARI G, DOLLAR P, et al. Mask R-CNN[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2): 386-397.
[35] LAW H, DENG J. CornerNet: Detecting objects as paired keypoints[J]. International Journal of Computer Vision, 2020, 128(3): 642-656.
[36] HAO Z K, LIU Y, QIN H W, et al. Scale-aware face detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE Press, 2017: 1913-1922.
[37] SONG G L, LIU Y, JIANG M, et al. Beyond trade-off: Accelerate FCN-based face detector with higher accuracy[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 7756-7764.
[38] LIU Y, LI H Y, YAN J J, et al. Recurrent scale approximation for object detection in CNN[C]//2017 IEEE International Conference on Computer Vision(ICCV). Piscataway: IEEE Press, 2017: 571-579.
[39] ZENG X Y, OUYANG W L, YAN J J, et al. Crafting GBD-net for object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(9): 2109-2123.
[40] LI H Y, LIU Y, OUYANG W L, et al. Zoom out-and-in network with map attention decision for region proposal and object detection[J]. International Journal of Computer Vision, 2019, 127(3): 225-238.
[41] SONG G L, LIU Y, WANG X G. Revisiting the sibling head in object detector[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE Press, 2020: 11560-11569.
[42] FU C Y, SHVETS M, BERG A C. RetinaMask: Learning to predict masks improves state-of-the-art single-shot detection for free[DB/OL]. arXiv preprint: 1901.03353, 2019.
[43] LEE Y, PARK J. CenterMask: Real-time anchor-free instance segmentation[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE Press, 2020: 13903-13912.
[44] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 7132-7141.
[45] WOO S, PARK J, LEE J Y, et al. CBAM: Convolutional block attention module[M]//Computer Vision-ECCV 2018. Cham: Springer International Publishing, 2018: 3-19.
[46] ZHU X Z, CHENG D Z, ZHANG Z, et al. An empirical study of spatial attention mechanisms in deep networks[C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV). Piscataway: IEEE Press, 2019: 6687-6696.
[47] QIN Z, LI Z M, ZHANG Z N, et al. ThunderNet: Towards real-time generic object detection on mobile devices[C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV). Piscataway: IEEE Press, 2019: 6717-6726.
[48] HU J, SHEN L, ALBANIE S, et al. Gather-excite: Exploiting feature context in convolutional neural networks[C]//Advances in Neural Information Processing Systems, 2018: 9401-9411.
[49] CHEN L, ZHANG H W, XIAO J, et al. SCA-CNN: Spatial and channel-wise attention in convolutional networks for image captioning[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE Press, 2017: 6298-6306.
[50] SANDLER M, HOWARD A, ZHU M L, et al. MobileNetV2: Inverted residuals and linear bottlenecks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 4510-4520.
[51] RUSSAKOVSKY O, DENG J, SU H, et al. ImageNet large scale visual recognition challenge[J]. International Journal of Computer Vision, 2015, 115(3): 211-252.
[52] REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149.
[53] LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE Press, 2017: 936-944.
[54] REDMON J, FARHADI A. YOLOv3: An incremental improvement[DB/OL]. arXiv preprint: 1804.02767, 2018.
[55] BOCHKOVSKIY A, WANG C Y, LIAO H Y MARK. YOLOv4: Optimal speed and accuracy of object detection[DB/OL]. arXiv preprint: 2004.10934, 2014.

Options

文章导航

模态框（Modal）标题

摘要

本文引用格式

Abstract

参考文献