无人机视觉引导对接过程中的协同目标检测

doi:10.7527/S1000-6893.2020.24854

Abstract

Abstract: Autonomous aerial recovery of UAV is a future development trend, and automatic detection of aerial vehicles is one of the key technologies to realize vision-guided recovery. At present, the research on the detection of aerial related objects is limited to individual objects, and the information between correlated objects is not fully utilized. For the problem of related object detection in high-dynamic aerial docking, this paper proposes a single-stage fast cooperative algorithm for detection of the master and the mount, including detection of sibling independent head of related category, detection of mask enhancement of related category, and constraints on consistency of features of related categories. These modules can improve the detection performance jointly. Experiments show that in the test dataset, the algorithm can obtain a 4.3% increase of the average precision of compared with YOLOv4, and can obtain a 31.6% increase of the average precision compared with YOLOv3-Tiny. At the same time, this algorithm has been applied to the high dynamic aerial docking project of MBZIRC2020 to achieve online real-time processing of airborne images, and our team won the championship.

Key words: UAV autonomous recovery, visual guided, object detection, related objects, convolutional neural networks

CLC Number:

V279

WANG Hui, JIA Zikai, JIN Ren, LIN Defu, FAN Junfang, XU Chao. Cooperative object detection in UAV-based vision-guided docking[J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(1): 324854-324854.

References

[1] 邹立岩, 张明智, 荣明. 智能无人机集群概念及主要发展趋势分析[J]. 战术导弹技术, 2019(5): 1-11, 43. ZOU L Y, ZHANG M Z, RONG M. Analysis of intelligent unmanned aircraft systems swarm concept and main development trend[J]. Tactical Missile Technology, 2019(5): 1-11, 43(in Chinese).
[2] 贾永楠, 田似营, 李擎. 无人机集群研究进展综述[J]. 航空学报, 2020, 41(S1): 723738. JIA Y N, TIAN S Y, LI Q. Recent development of unmanned aerial vehicle swarms[J]. Acta Aeronautica et Astronautica Sinica, 2020, 41(S1): 723738(in Chinese).
[3] 王祥科, 刘志宏, 丛一睿, 等. 小型固定翼无人机集群综述和未来发展[J]. 航空学报, 2020, 41(4): 023732. WANG X K, LIU Z H, CONG Y R, et al. Miniature fixed-wing UAV swarms: Review and outlook[J]. Acta Aeronautica et Astronautica Sinica, 2020, 41(4): 023732(in Chinese).
[4] 牛轶峰, 肖湘江, 柯冠岩. 无人机集群作战概念及关键技术分析[J]. 国防科技, 2013, 34(5): 37-43. NIU Y F, XIAO X J, KE G Y. Operation concept and key techniques of unmanned aerial vehicle swarms[J]. National Defense Science & Technology, 2013, 34(5): 37-43(in Chinese).
[5] 李俊国. 蜂群式固定翼无人机空基回收系统设计及动力学分析[D]. 哈尔滨: 哈尔滨工业大学, 2017. LI J G. Design and dynamics analysis of air base recovery system for colony type fixed wing UAV[D]. Harbin: Harbin Institute of Technology, 2017(in Chinese).
[6] VENDRA S, CAMPA G, NAPOLITANO M R, et al. Addressing corner detection issues for machine vision based UAV aerial refueling[J]. Machine Vision and Applications, 2007, 18(5): 261-273.
[7] DELL’AQUILA R V, CAMPA G, NAPOLITANO M R, et al. Real-time machine-vision-based position sensing system for UAV aerial refueling[J]. Journal of Real-Time Image Processing, 2007, 1(3): 213-224.
[8] HARRIS C, STEPHENS M. A combined corner and edge detector[C]//Proceedings ofthe Alvey Vision Conference, 1988: 147-151.
[9] SMITH S M, BRADY J M. Susan-a new approach to low level image processing[J]. International Journal of Computer Vision, 1997, 23(1): 45-78.
[10] CAMPA G, MAMMARELLA M, NAPOLITANO M R, et al. A comparison of pose estimation algorithms for machine vision based aerial refueling for UAVs[C]//2006 14th Mediterranean Conference on Control and Automation. Piscataway: IEEE Press, 2006: 1-6.
[11] FRAVOLINI M L, CAMPA G, NAPOLITANO M R. Evaluation of machine vision algorithms for autonomous aerial refueling for unmanned aerial vehicles[J]. Journal of Aerospace Computing, Information, and Communication, 2007, 4(9): 968-985.
[12] MASH R, BORGHETTI B, PECARINA J. Improved aircraft recognition for aerial refueling through data augmentation in convolutional neural networks[C]//Advances in Visual Computing, 2016.
[13] WANG X F, KONG X W, ZHI J H, et al. Real-time drogue recognition and 3D locating for UAV autonomous aerial refueling based on monocular machine vision[J]. Chinese Journal of Aeronautics, 2015, 28(6): 1667-1675.
[14] YIN Y J, WANG X G, XU D, et al. Robust visual detection-learning-tracking framework for autonomous aerial refueling of UAVs[J]. IEEE Transactions on Instrumentation and Measurement, 2016, 65(3): 510-521.
[15] CHEN A G, WANG B X, YANG C T, et al. Research on drogue detection algorithm for aerial refueling(IEEE/CSAA GNCC)[C]//2018 IEEE CSAA Guidance, Navigation and Control Conference(CGNCC). Piscataway: IEEE Press, 2018: 1-4.
[16] TU Z W, BAI X. Auto-context and its application to high-level vision tasks and 3D brain image segmentation[C]//IEEE Transactions on Pattern Analysis and Machine Intelligence. Piscataway: IEEE Press, 2010: 1744-1757.
[17] GALLEGUILLOS C, RABINOVICH A, BELONGIE S. Object categorization using co-occurrence, location and appearance[C]//2008 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2008: 1-8.
[18] TORRALBA, MURPHY, FREEMAN, et al. Context-based vision system for place and object recognition[C]//Proceedings Ninth IEEE International Conference on Computer Vision. Piscataway: IEEE Press, 2003: 273-280.
[19] FELZENSZWALB P F, GIRSHICK R B, MCALLESTER D, et al. Object detection with discriminatively trained part-based models[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(9): 1627-1645.
[20] BIEDERMAN I, MEZZANOTTE R J, RABINOWITZ J C. Scene perception: Detecting and judging objects undergoing relational violations[J]. Cognitive Psychology, 1982, 14(2): 143-177.
[21] CHOI M J, TORRALBA A, WILLSKY A S. A tree-based context model for object recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(2): 240-252.
[22] MOTTAGHI R, CHEN X J, LIU X B, et al. The role of context for object detection and semantic segmentation in the wild[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2014: 891-898.
[23] HU H, GU J Y, ZHANG Z, et al. Relation networks for object detection[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 3588-3597.
[24] 张咪, 赵勇, 布树辉, 等. 基于阶层标识的无人机自主精准降落系统[J]. 航空学报, 2018, 39(10): 322150. ZHANG M, ZHAO Y, BU S H, et al. Multi-level marker based autonomous landing system for UAVs[J]. Acta Aeronautica et Astronautica Sinica, 2018, 39(10): 322150(in Chinese).
[25] 高嘉瑜, 袁苏哲, 景鑫, 等. 基于AprilTag二维码的无人机着陆引导方法[J]. 现代导航, 2020, 11(1): 20-25. GAO J Y, YUAN S Z, JING X, et al. Method of UAV position based on cooperative two-dimension code[J]. Modern Navigation, 2020, 11(1): 20-25(in Chinese).
[26] 赵文一. 无人机视觉辅助自主降落系统研究[D]. 哈尔滨: 哈尔滨工业大学, 2018. ZHAO W Y. Research on vision-based autonomous landing system of UAV[D]. Harbin: Harbin Institute of Technology, 2018(in Chinese).
[27] FALANGA D, ZANCHETTIN A, SIMOVIC A, et al. Vision-based autonomous quadrotor landing on a moving platform[C]//2017 IEEE International Symposium on Safety, Security and Rescue Robotics(SSRR). Piscataway: IEEE Press, 2017: 200-207.
[28] BACA T, STEPAN P, SASKA M. Autonomous landing on a moving car with unmanned aerial vehicle[C]//2017 European Conference on Mobile Robots(ECMR). Piscataway: IEEE Press, 2017: 1-6.
[29] TRUONG N Q, NGUYEN P H, NAM S H, et al. Deep learning-based super-resolution reconstruction and marker detection for drone landing[J]. IEEE Access, 2019, 7: 61639-61655.
[30] SUN S Y, YIN Y J, WANG X G, et al. Robust visual detection and tracking strategies for autonomous aerial refueling of UAVs[J]. IEEE Transactions on Instrumentation and Measurement, 2019, 68(12): 4640-4652.
[31] 江波, 屈若锟, 李彦冬, 等. 基于深度学习的无人机航拍目标检测研究综述[J]. 航空学报, 2021, 42(4): 524519. JIANG B, QU R K, LI Y D, et al. Object detection in UAV imagery based on deep learning: Review[J]. Acta Aeronautica et Astronautica Sinica, 2021, 42(4): 524519(in Chinese).
[32] 方路平, 何杭江, 周国民. 目标检测算法研究综述[J]. 计算机工程与应用, 2018, 54(13): 11-18, 33. FANG L P, HE H J, ZHOU G M. Research overview of object detection methods[J]. Computer Engineering and Applications, 2018, 54(13): 11-18, 33(in Chinese).
[33] TIAN Z, SHEN C H, CHEN H, et al. FCOS: Fully convolutional one-stage object detection[C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV). Piscataway: IEEE Press, 2019: 9626-9635.
[34] HE K M, GKIOXARI G, DOLLAR P, et al. Mask R-CNN[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2): 386-397.
[35] LAW H, DENG J. CornerNet: Detecting objects as paired keypoints[J]. International Journal of Computer Vision, 2020, 128(3): 642-656.
[36] HAO Z K, LIU Y, QIN H W, et al. Scale-aware face detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE Press, 2017: 1913-1922.
[37] SONG G L, LIU Y, JIANG M, et al. Beyond trade-off: Accelerate FCN-based face detector with higher accuracy[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 7756-7764.
[38] LIU Y, LI H Y, YAN J J, et al. Recurrent scale approximation for object detection in CNN[C]//2017 IEEE International Conference on Computer Vision(ICCV). Piscataway: IEEE Press, 2017: 571-579.
[39] ZENG X Y, OUYANG W L, YAN J J, et al. Crafting GBD-net for object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(9): 2109-2123.
[40] LI H Y, LIU Y, OUYANG W L, et al. Zoom out-and-in network with map attention decision for region proposal and object detection[J]. International Journal of Computer Vision, 2019, 127(3): 225-238.
[41] SONG G L, LIU Y, WANG X G. Revisiting the sibling head in object detector[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE Press, 2020: 11560-11569.
[42] FU C Y, SHVETS M, BERG A C. RetinaMask: Learning to predict masks improves state-of-the-art single-shot detection for free[DB/OL]. arXiv preprint: 1901.03353, 2019.
[43] LEE Y, PARK J. CenterMask: Real-time anchor-free instance segmentation[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE Press, 2020: 13903-13912.
[44] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 7132-7141.
[45] WOO S, PARK J, LEE J Y, et al. CBAM: Convolutional block attention module[M]//Computer Vision-ECCV 2018. Cham: Springer International Publishing, 2018: 3-19.
[46] ZHU X Z, CHENG D Z, ZHANG Z, et al. An empirical study of spatial attention mechanisms in deep networks[C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV). Piscataway: IEEE Press, 2019: 6687-6696.
[47] QIN Z, LI Z M, ZHANG Z N, et al. ThunderNet: Towards real-time generic object detection on mobile devices[C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV). Piscataway: IEEE Press, 2019: 6717-6726.
[48] HU J, SHEN L, ALBANIE S, et al. Gather-excite: Exploiting feature context in convolutional neural networks[C]//Advances in Neural Information Processing Systems, 2018: 9401-9411.
[49] CHEN L, ZHANG H W, XIAO J, et al. SCA-CNN: Spatial and channel-wise attention in convolutional networks for image captioning[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE Press, 2017: 6298-6306.
[50] SANDLER M, HOWARD A, ZHU M L, et al. MobileNetV2: Inverted residuals and linear bottlenecks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2018: 4510-4520.
[51] RUSSAKOVSKY O, DENG J, SU H, et al. ImageNet large scale visual recognition challenge[J]. International Journal of Computer Vision, 2015, 115(3): 211-252.
[52] REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149.
[53] LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway: IEEE Press, 2017: 936-944.
[54] REDMON J, FARHADI A. YOLOv3: An incremental improvement[DB/OL]. arXiv preprint: 1804.02767, 2018.
[55] BOCHKOVSKIY A, WANG C Y, LIAO H Y MARK. YOLOv4: Optimal speed and accuracy of object detection[DB/OL]. arXiv preprint: 2004.10934, 2014.

Cooperative object detection in UAV-based vision-guided docking

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	Xudong LUO, Yiquan WU, Jinlin CHEN. Research progress on deep learning methods for object detection and semantic segmentation in UAV aerial images [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(6): 28822-028822.
[2]	Jiqiang GAN, Xiaoping WANG. Surface defect detection of fiber placement based on virtual sample generation [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(1): 428624-428624.
[3]	Zhiqiang FENG, Zhijun XIE, Zhengwei BAO, Kewei CHEN. Real⁃time dense small object detection algorithm for UAV based on improved YOLOv5 [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2023, 44(7): 327106-327106.
[4]	Lei HE, Weiqi QIAN, Kangsheng DONG, Xian YI, Congcong CHAI. Aerodynamic characteristics modeling of iced airfoil based on convolution neural networks [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2023, 44(5): 126434-126434.
[5]	Guotao MAO, Tianmin DENG, Nanjing YU. Object detection in UAV images based on multi-scale split attention [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2023, 44(5): 326738-326738.
[6]	Yubin YUAN, Yiquan WU, Langyue ZHAO, Jinlin CHEN, Qichang ZHAO. Research progress of UAV aerial video multi⁃object detection and tracking based on deep learning [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(18): 28334-028334.
[7]	Yucun SONG, Quanbo GE, Junlong ZHU, Zhenyu LU. Improved YOLOX object detection algorithm based on gradient difference adaptive learning rate optimization [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(14): 327951-327951.
[8]	Zihao LI, Zhengping WANG, Yuntao HE. Aerial-photography dense small target detection algorithm based on adaptive cooperative attention mechanism [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(13): 327944-327944.
[9]	LIU Fang, HAN Xiao. Adaptive aerial object detection based on multi-scale deep learning [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(5): 325270-325270.
[10]	LI Hongguang, YU Ruonan, DING Wenrui. Research development of small object traching based on deep learning [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2021, 42(7): 24691-024691.
[11]	DING Wenrui, LIU Chunlei, LI Yue, ZHANG Baochang. Binary convolutional neural network: Review [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2021, 42(6): 24504-024504.
[12]	WANG Zelin, JI Ritian, HUI Xinyu, DING Chen, WANG Hui, BAI Junqiang. L-shaped directional heat transfer based on deep learning [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2021, 42(6): 124242-124242.
[13]	JIANG Bo, QU Ruokun, LI Yandong, LI Chenglong. Object detection in UAV imagery based on deep learning: Review [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2021, 42(4): 524519-524519.
[14]	ZHANG Kai, WANG Kaidi, YANG Xi, LI Shaoyi, WANG Xiaotian. Anti-interference recognition algorithm based on DNET for infrared aerial target [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2021, 42(2): 324223-324223.
[15]	SONG Chuang, ZHAO Jiajia, WANG Kang, LIANG Xinkai. A survey of few shot learning based on intelligent perception [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2020, 41(S1): 723756-723756.