Cooperative object detection in UAV-based vision-guided docking

WANG Hui; JIA Zikai; JIN Ren; LIN Defu; FAN Junfang; XU Chao

doi:10.7527/S1000-6893.2020.24854

ACTA AERONAUTICAET ASTRONAUTICA SINICA >

2022 , Vol. 43 >Issue 1: 324854 - 324854

DOI: https://doi.org/10.7527/S1000-6893.2020.24854

Electronics and Electrical Engineering and Control

Cooperative object detection in UAV-based vision-guided docking

WANG Hui ,
JIA Zikai ,
JIN Ren ,
LIN Defu ,
FAN Junfang ,
XU Chao

Expand

1. School of Aerospace Engineering, Beijing Institute of Technology, Beijing 100081, China;
2. Beijing Key Laboratory of UAV Autonomous Control, Beijing Institute of Technology, Beijing 100081, China;
3. Beijing Key Laboratory of High-Dynamic Navigation Technology, Beijing Information Science and Technology University, Beijing 100085, China;
4. Beijing Institute of Special Mechanic-Electric, Beijing 100012, China

Received date: 2020-10-12

Revised date: 2020-12-10

Online published: 2020-12-03

Supported by

National Natural Science Foundation of China(U1613225); Funded Project of Beijing Key Laboratory of High Dynamic Navigation Technology of Beijing Information Science and Technology University(HDN2020105); Open Fund Project of Beijing Key Laboratory of High Dynamic Navigation Technology of Beijing Information Science and Technology University(HDN2020101)

Fold

Abstract

Autonomous aerial recovery of UAV is a future development trend, and automatic detection of aerial vehicles is one of the key technologies to realize vision-guided recovery. At present, the research on the detection of aerial related objects is limited to individual objects, and the information between correlated objects is not fully utilized. For the problem of related object detection in high-dynamic aerial docking, this paper proposes a single-stage fast cooperative algorithm for detection of the master and the mount, including detection of sibling independent head of related category, detection of mask enhancement of related category, and constraints on consistency of features of related categories. These modules can improve the detection performance jointly. Experiments show that in the test dataset, the algorithm can obtain a 4.3% increase of the average precision of compared with YOLOv4, and can obtain a 31.6% increase of the average precision compared with YOLOv3-Tiny. At the same time, this algorithm has been applied to the high dynamic aerial docking project of MBZIRC2020 to achieve online real-time processing of airborne images, and our team won the championship.

Key words： UAV autonomous recovery; visual guided; object detection; related objects; convolutional neural networks

Cite this article

WANG Hui , JIA Zikai , JIN Ren , LIN Defu , FAN Junfang , XU Chao . Cooperative object detection in UAV-based vision-guided docking[J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022 , 43(1) : 324854 -324854 . DOI: 10.7527/S1000-6893.2020.24854

References

[1] 邹立岩, 张明智, 荣明. 智能无人机集群概念及主要发展趋势分析[J]. 战术导弹技术, 2019(5): 1-11, 43. ZOU L Y, ZHANG M Z, RONG M. Analysis of intelligent unmanned aircraft systems swarm concept and main development trend[J]. Tactical Missile Technology, 2019(5): 1-11, 43(in Chinese).
[2] 贾永楠, 田似营, 李擎. 无人机集群研究进展综述[J]. 航空学报, 2020, 41(S1): 723738. JIA Y N, TIAN S Y, LI Q. Recent development of unmanned aerial vehicle swarms[J]. Acta Aeronautica et Astronautica Sinica, 2020, 41(S1): 723738(in Chinese).
[3] 王祥科, 刘志宏, 丛一睿, 等. 小型固定翼无人机集群综述和未来发展[J]. 航空学报, 2020, 41(4): 023732. WANG X K, LIU Z H, CONG Y R, et al. Miniature fixed-wing UAV swarms: Review and outlook[J]. Acta Aeronautica et Astronautica Sinica, 2020, 41(4): 023732(in Chinese).
[4] 牛轶峰, 肖湘江, 柯冠岩. 无人机集群作战概念及关键技术分析[J]. 国防科技, 2013, 34(5): 37-43. NIU Y F, XIAO X J, KE G Y. Operation concept and key techniques of unmanned aerial vehicle swarms[J]. National Defense Science & Technology, 2013, 34(5): 37-43(in Chinese).
[5] 李俊国. 蜂群式固定翼无人机空基回收系统设计及动力学分析[D]. 哈尔滨: 哈尔滨工业大学, 2017. LI J G. Design and dynamics analysis of air base recovery system for colony type fixed wing UAV[D]. Harbin: Harbin Institute of Technology, 2017(in Chinese).
[6] VENDRA S, CAMPA G, NAPOLITANO M R, et al. Addressing corner detection issues for machine vision based UAV aerial refueling[J]. Machine Vision and Applications, 2007, 18(5): 261-273.
[7] DELL’AQUILA R V, CAMPA G, NAPOLITANO M R, et al. Real-time machine-vision-based position sensing system for UAV aerial refueling[J]. Journal of Real-Time Image Processing, 2007, 1(3): 213-224.
[8] HARRIS C, STEPHENS M. A combined corner and edge detector[C]//Proceedings ofthe Alvey Vision Conference, 1988: 147-151.
[9] SMITH S M, BRADY J M. Susan-a new approach to low level image processing[J]. International Journal of Computer Vision, 1997, 23(1): 45-78.
[10] CAMPA G, MAMMARELLA M, NAPOLITANO M R, et al. A comparison of pose estimation algorithms for machine vision based aerial refueling for UAVs[C]//2006 14th Mediterranean Conference on Control and Automation. Piscataway: IEEE Press, 2006: 1-6.
[11] FRAVOLINI M L, CAMPA G, NAPOLITANO M R. Evaluation of machine vision algorithms for autonomous aerial refueling for unmanned aerial vehicles[J]. Journal of Aerospace Computing, Information, and Communication, 2007, 4(9): 968-985.
[12] MASH R, BORGHETTI B, PECARINA J. Improved aircraft recognition for aerial refueling through data augmentation in convolutional neural networks[C]//Advances in Visual Computing, 2016.
[13] WANG X F, KONG X W, ZHI J H, et al. Real-time drogue recognition and 3D locating for UAV autonomous aerial refueling based on monocular machine vision[J]. Chinese Journal of Aeronautics, 2015, 28(6): 1667-1675.
[14] YIN Y J, WANG X G, XU D, et al. Robust visual detection-learning-tracking framework for autonomous aerial refueling of UAVs[J]. IEEE Transactions on Instrumentation and Measurement, 2016, 65(3): 510-521.
[15] CHEN A G, WANG B X, YANG C T, et al. Research on drogue detection algorithm for aerial refueling(IEEE/CSAA GNCC)[C]//2018 IEEE CSAA Guidance, Navigation and Control Conference(CGNCC). Piscataway: IEEE Press, 2018: 1-4.
[16] TU Z W, BAI X. Auto-context and its application to high-level vision tasks and 3D brain image segmentation[C]//IEEE Transactions on Pattern Analysis and Machine Intelligence. Piscataway: IEEE Press, 2010: 1744-1757.
[17] GALLEGUILLOS C, RABINOVICH A, BELONGIE S. Object categorization using co-occurrence, location and appearance[C]//2008 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2008: 1-8.
[18] TORRALBA, MURPHY, FREEMAN, et al. Context-based vision system for place and object recognition[C]//Proceedings Ninth IEEE International Conference on Computer Vision. Piscataway: IEEE Press, 2003: 273-280.
[19] FELZENSZWALB P F, GIRSHICK R B, MCALLESTER D, et al. Object detection with discriminatively trained part-based models[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(9): 1627-1645.
[20] BIEDERMAN I, MEZZANOTTE R J, RABINOWITZ J C. Scene perception: Detecting and judging objects undergoing relational violations[J]. Cognitive Psychology, 1982, 14(2): 143-177.
[21] CHOI M J, TORRALBA A, WILLSKY A S. A tree-based context model for object recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(2): 240-252.
[22] MOTTAGHI R, CHEN X J, LIU X B, et al. The role of context for object detection and semantic segmentation in the wild[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2014: 891-898.
[23] HU H, GU J Y, ZHANG Z, et al. Relation networks for object detection[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 3588-3597.
[24] 张咪, 赵勇, 布树辉, 等. 基于阶层标识的无人机自主精准降落系统[J]. 航空学报, 2018, 39(10): 322150. ZHANG M, ZHAO Y, BU S H, et al. Multi-level marker based autonomous landing system for UAVs[J]. Acta Aeronautica et Astronautica Sinica, 2018, 39(10): 322150(in Chinese).
[25] 高嘉瑜, 袁苏哲, 景鑫, 等. 基于AprilTag二维码的无人机着陆引导方法[J]. 现代导航, 2020, 11(1): 20-25. GAO J Y, YUAN S Z, JING X, et al. Method of UAV position based on cooperative two-dimension code[J]. Modern Navigation, 2020, 11(1): 20-25(in Chinese).
[26] 赵文一. 无人机视觉辅助自主降落系统研究[D]. 哈尔滨: 哈尔滨工业大学, 2018. ZHAO W Y. Research on vision-based autonomous landing system of UAV[D]. Harbin: Harbin Institute of Technology, 2018(in Chinese).
[27] FALANGA D, ZANCHETTIN A, SIMOVIC A, et al. Vision-based autonomous quadrotor landing on a moving platform[C]//2017 IEEE International Symposium on Safety, Security and Rescue Robotics(SSRR). Piscataway: IEEE Press, 2017: 200-207.
[28] BACA T, STEPAN P, SASKA M. Autonomous landing on a moving car with unmanned aerial vehicle[C]//2017 European Conference on Mobile Robots(ECMR). Piscataway: IEEE Press, 2017: 1-6.
[29] TRUONG N Q, NGUYEN P H, NAM S H, et al. Deep learning-based super-resolution reconstruction and marker detection for drone landing[J]. IEEE Access, 2019, 7: 61639-61655.
[30] SUN S Y, YIN Y J, WANG X G, et al. Robust visual detection and tracking strategies for autonomous aerial refueling of UAVs[J]. IEEE Transactions on Instrumentation and Measurement, 2019, 68(12): 4640-4652.
[31] 江波, 屈若锟, 李彦冬, 等. 基于深度学习的无人机航拍目标检测研究综述[J]. 航空学报, 2021, 42(4): 524519. JIANG B, QU R K, LI Y D, et al. Object detection in UAV imagery based on deep learning: Review[J]. Acta Aeronautica et Astronautica Sinica, 2021, 42(4): 524519(in Chinese).
[32] 方路平, 何杭江, 周国民. 目标检测算法研究综述[J]. 计算机工程与应用, 2018, 54(13): 11-18, 33. FANG L P, HE H J, ZHOU G M. Research overview of object detection methods[J]. Computer Engineering and Applications, 2018, 54(13): 11-18, 33(in Chinese).
[33] TIAN Z, SHEN C H, CHEN H, et al. FCOS: Fully convolutional one-stage object detection[C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV). Piscataway: IEEE Press, 2019: 9626-9635.
[34] HE K M, GKIOXARI G, DOLLAR P, et al. Mask R-CNN[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2): 386-397.
[35] LAW H, DENG J. CornerNet: Detecting objects as paired keypoints[J]. International Journal of Computer Vision, 2020, 128(3): 642-656.
[36] HAO Z K, LIU Y, QIN H W, et al. Scale-aware face detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE Press, 2017: 1913-1922.
[37] SONG G L, LIU Y, JIANG M, et al. Beyond trade-off: Accelerate FCN-based face detector with higher accuracy[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 7756-7764.
[38] LIU Y, LI H Y, YAN J J, et al. Recurrent scale approximation for object detection in CNN[C]//2017 IEEE International Conference on Computer Vision(ICCV). Piscataway: IEEE Press, 2017: 571-579.
[39] ZENG X Y, OUYANG W L, YAN J J, et al. Crafting GBD-net for object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(9): 2109-2123.
[40] LI H Y, LIU Y, OUYANG W L, et al. Zoom out-and-in network with map attention decision for region proposal and object detection[J]. International Journal of Computer Vision, 2019, 127(3): 225-238.
[41] SONG G L, LIU Y, WANG X G. Revisiting the sibling head in object detector[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE Press, 2020: 11560-11569.
[42] FU C Y, SHVETS M, BERG A C. RetinaMask: Learning to predict masks improves state-of-the-art single-shot detection for free[DB/OL]. arXiv preprint: 1901.03353, 2019.
[43] LEE Y, PARK J. CenterMask: Real-time anchor-free instance segmentation[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE Press, 2020: 13903-13912.
[44] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 7132-7141.
[45] WOO S, PARK J, LEE J Y, et al. CBAM: Convolutional block attention module[M]//Computer Vision-ECCV 2018. Cham: Springer International Publishing, 2018: 3-19.
[46] ZHU X Z, CHENG D Z, ZHANG Z, et al. An empirical study of spatial attention mechanisms in deep networks[C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV). Piscataway: IEEE Press, 2019: 6687-6696.
[47] QIN Z, LI Z M, ZHANG Z N, et al. ThunderNet: Towards real-time generic object detection on mobile devices[C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV). Piscataway: IEEE Press, 2019: 6717-6726.
[48] HU J, SHEN L, ALBANIE S, et al. Gather-excite: Exploiting feature context in convolutional neural networks[C]//Advances in Neural Information Processing Systems, 2018: 9401-9411.
[49] CHEN L, ZHANG H W, XIAO J, et al. SCA-CNN: Spatial and channel-wise attention in convolutional networks for image captioning[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE Press, 2017: 6298-6306.
[50] SANDLER M, HOWARD A, ZHU M L, et al. MobileNetV2: Inverted residuals and linear bottlenecks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 4510-4520.
[51] RUSSAKOVSKY O, DENG J, SU H, et al. ImageNet large scale visual recognition challenge[J]. International Journal of Computer Vision, 2015, 115(3): 211-252.
[52] REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149.
[53] LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE Press, 2017: 936-944.
[54] REDMON J, FARHADI A. YOLOv3: An incremental improvement[DB/OL]. arXiv preprint: 1804.02767, 2018.
[55] BOCHKOVSKIY A, WANG C Y, LIAO H Y MARK. YOLOv4: Optimal speed and accuracy of object detection[DB/OL]. arXiv preprint: 2004.10934, 2014.

Options

Outlines

模态框（Modal）标题

Abstract

Cite this article

References