基于多尺度分割注意力的无人机航拍图像目标检测算法

doi:10.7527/S1000-6893.2021.26738

Abstract

Abstract:

With the development of Unmanned Aerial Vehicle （UAV） remote sensing technology， UAV aerial image object detection has become a core technology in the field of UAV applications such as traffic planning， military reconnaissance and environmental monitoring. To overcome the problem of difficulty in feature extraction due to many instances of small objects and complex background in UAV images， this paper proposes an object detection algorithm for UAV aerial images based on multi-scale split attention， i.e.， MAS-YOLO. Firstly， the multi-scale split attention unit embedded in the bottleneck layer of the backbone net-work is used to establish the long-range dependency relationship between different scales of attention， so as to enhance the expression ability of key features and suppress the interference of background noise. Secondly， an adaptive weighted feature fusion method is designed， which dynamically optimizes the weight of each output feature layer and realize the deep fusion of shallow and deep features. Finally， experimental results on the VisDrone public data set show that the proposed method achieves 34.7% mean Average Precision （mAP）， which is 2.8% higher than that of the baseline algorithm YOLOv5， and can also significantly improve the performance of UAV image object detection in complex background.

Key words: unmanned aerial vehicle image, computer vision, object detection, attention mechanism, adaptive weighted feature fusion

CLC Number:

V279

Guotao MAO, Tianmin DENG, Nanjing YU. Object detection in UAV images based on multi-scale split attention[J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2023, 44(5): 326738-326738.

Figures/Tables 8

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Table 1

Table 2

Fig. 5

Fig. 6

References 24

1	江波，屈若锟，李彦冬，等. 基于深度学习的无人机航拍目标检测研究综述［J］. 航空学报， 2021， 42（4）： 524519.
	JIANG B， QU R K， LI Y D， et al. Object detection in UAV imagery based on deep learning： Review［J］. Acta Aeronautica et Astronautica Sinica， 2021， 42（4）： 524519 （in Chinese）.
2	REN S Q， HE K M， GIRSHICK R， et al. Faster R-CNN： Towards real-time object detection with region proposal networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（6）： 1137-1149.
3	HE K M， GKIOXARI G， DOLLÁR P， et al. Mask R-CNN［C］∥2017 IEEE International Conference on Computer Vision. Piscataway： IEEE Press， 2017： 2980-2988.
4	CAI Z W， VASCONCELOS N. Cascade R-CNN： Delving into high quality object detection［C］∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2018： 6154-6162.
5	LIU Y J， YANG F B， HU P. Small-object detection in UAV-captured images via multi-branch parallel feature pyramid networks［J］. IEEE Access， 2020， 8： 145740-145750.
6	LIN Q Z， DING Y， XU H， et al. ECascade-RCNN： Enhanced cascade RCNN for multi-scale object detection in UAV images［C］∥2021 7th International Conference on Automation， Robotics and Applications （ICARA）. Piscataway： IEEE Press， 2021： 268-272.
7	REDMON J， DIVVALA S， GIRSHICK R， et al. You only look once： Unified， real-time object detection［C］∥2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2016： 779-788.
8	LIU W， ANGUELOV D， ERHAN D， et al. SSD： Single shot MultiBox detector［C］∥Proceedings of the 14th European Conference on Computer Vision （ECCV）. Berlin： Springer， 2016： 21-37.
9	LIN T Y， GOYAL P， GIRSHICK R， et al. Focal loss for dense object detection［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2020， 42（2）： 318-327.
10	ZHANG Z Y， LIU Y P， LIU T C， et al. DAGN： A real-time UAV remote sensing image vehicle detection framework［J］. IEEE Geoscience and Remote Sensing Letters， 2020， 17（11）： 1884-1888.
11	WANG X R， LI W H， GUO W， et al. SPB-YOLO： An efficient real-time detector for unmanned aerial vehicle images［C］∥2021 International Conference on Artificial Intelligence in Information and Communication （ICAIIC）. Piscataway： IEEE Press， 2021： 99-104.
12	LIU S， QI L， QIN H F， et al. Path aggregation network for instance segmentation［C］∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2018： 8759-8768.
13	裴伟，许晏铭，朱永英，等. 改进的SSD航拍目标检测方法［J］. 软件学报， 2019， 30（3）： 738-758.
	PEI W， XU Y M， ZHU Y Y， et al. The target detection method of aerial photography images with improved SSD［J］. Journal of Software， 2019， 30（3）： 738-758 （in Chinese）.
14	赵辉，李志伟，张天琪. 基于注意力机制的单发多框检测器算法［J］. 电子与信息学报， 2021， 43（7）： 2096-2104.
	ZHAO H， LI Z W， ZHANG T Q. Attention based single shot multibox detector［J］. Journal of Electronics ＆ Information Technology， 2021， 43（7）： 2096-2104 （in Chinese）.
15	WOO S， PARK J， LEE J Y， et al. CBAM： Convolutional block attention module［C］∥Proceedings of the 15th European Conference on Computer Vision （ECCV）. Berlin： Springer， 2018： 1352-1368.
16	王美华，吴振鑫，周祖光. 基于注意力改进CBAM的农作物病虫害细粒度识别研究［J］. 农业机械学报， 2021， 52（4）： 239-247.
	WANG M H， WU Z X， ZHOU Z G. Fine-grained identification research of crop pests and diseases based on improved CBAM via attention［J］. Transactions of the Chinese Society for Agricultural Machinery， 2021， 52（4）： 239-247 （in Chinese）.
17	LIU S T， HUANG D， WANG Y H. Learning spatial fusion for single-shot object detection［DB/OL］. arXiv preprint： 1911.09516， 2019.
18	ZHU P F， WEN L Y， DU D W， et al. Vision meets drones： Past， present and future［DB/OL］. arXiv preprint： 2001.06303， 2020.
19	YU W P， YANG T， CHEN C. Towards resolving the challenge of long-tail distribution in UAV images for object detection［C］∥2021 IEEE Winter Conference on Applications of Computer Vision. Piscataway： IEEE Press， 2021： 3257-3266.
20	ALBABA B M， OZER S. S_yNet： An ensemble network for object detection in UAV images［C］∥2020 25th International Conference on Pattern Recognition （ICPR）. Piscataway： IEEE Press， 2020： 10227-10234.
21	ALI S， SIDDIQUE A， ATEŞ H F， et al. Improved YOLOv4 for aerial object detection［C］∥2021 29th Signal Processing and Communications Applications Conference （SIU）. Piscataway： IEEE Press， 2021： 1-4.
22	CAO Y R， HE Z J， WANG L J， et al. VisDrone-DET2021： The vision meets drone object detection challenge results［C］∥2021 IEEE/CVF International Conference on Computer Vision Workshops （ICCVW）. Piscataway： IEEE Press， 2021： 2847-2854.
23	DU D W， ZHU P F， WEN L Y， et al. VisDrone-DET2019： The vision meets drone object detection in image challenge results［C］∥2019 IEEE/CVF International Conference on Computer Vision Workshop （ICCVW）. Piscataway： IEEE Press， 2019： 213-226.
24	ZHAO H P， ZHOU Y， ZHANG L， et al. Mixed YOLOv3-LITE： A lightweight real-time object detection method［J］. Sensors （Basel， Switzerland）， 2020， 20（7）： 1861.

方法	mAP	模型规模/MB	参数量/B	FLOP/G
基线算法	31.9	89	46.68	114.4
基线算法+MSAU	34.1	104	54.21	140.9
基线算法+AWF	32.8	94	48.89	119.8
MSA-YOLO	34.7	108	56.28	146.1

方法	骨干网络	AP/%										mAP/%
方法	骨干网络	行人	人	自行车	汽车	火车	卡车	三轮车	遮阳蓬三轮车	公共汽车	摩托车	mAP/%
Faster R-CNN^［19］	ResNet-50	21.4	15.6	6.7	51.7	29.5	19.0	13.1	7.7	31.4	20.7	21.7
Faster R-CNN^［19］	ResNet-101	20.9	14.8	7.3	51.0	29.7	19.5	14.0	8.8	30.5	21.2	21.8
Cascade R-CNN^［19］	ResNet-50	22.2	14.8	7.6	54.6	31.5	21.6	14.8	8.6	34.9	21.4	23.2
RetinaNet^［19］	ResNet-50	13.0	7.9	1.4	45.5	19.9	11.5	6.3	4.2	17.8	11.8	13.9
CenterNet^［20］	Hourglass-104	22.6	20.6	14.6	59.7	24.0	21.3	20.1	17.4	37.9	23.7	26.2
YOLOv4^［21］	CSPDarknet	24.8	12.6	8.6	64.3	22.4	22.7	11.4	7.6	44.3	21.7	30.7
DMNet^［22］	ResNet-50	28.5	20.4	15.9	56.8	37.9	30.1	22.6	14.0	47.1	29.2	30.3
DBAI-Det^［23］	ResNeXt-101	36.7	12.8	14.7	47.4	38.0	41.4	23.4	16.9	31.9	16.6	28.0
CDNet^［22］	ResNeXt-101	35.6	19.2	13.8	55.8	42.1	38.2	33.0	25.4	49.5	29.3	34.2
HR-Cascade++^［22］	HRNet-W40	32.6	17.3	11.1	54.7	42.4	35.3	32.7	24.1	46.5	28.2	32.5
MSC-CenterNet^［22］	Hourglass-104	33.7	15.2	12.1	55.2	40.5	34.1	29.2	21.6	42.2	27.5	31.1
YOLOv3-LITE^［24］	Darknet-53	34.5	23.4	7.9	70.8	31.3	21.9	15.3	6.2	40.9	32.7	28.5
MSA-YOLO	CSPDarknet	33.4	17.3	11.2	76.8	41.5	41.4	14.8	18.4	60.9	31.0	34.7

[1]	Baohui JIA, Fan JIANG, Yuxin WANG, Du WANG. Fault diagnosis method based on civil aircraft maintenance text data [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2023, 44(5): 326598-326598.
[2]	Ping WANG, Hui FU, Guili XU. Camera pose estimation and corresponding points matching based on rotation search [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2023, 44(2): 326695-326695.
[3]	ZHONG Mengfan, PEI Jihong. Image stitching enhancement method with symmetrical constraint of local feature points [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(S1): 726948-726948.
[4]	LYU Shuaishuai, YANG Yu, WANG Binwen, YIN Chenfei. A crack identification method of aircraft structure based on improved FaceNet [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(6): 525463-525463.
[5]	LIU Fang, HAN Xiao. Adaptive aerial object detection based on multi-scale deep learning [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(5): 325270-325270.
[6]	WANG Zi, SUN Xiaoliang, LI Zhang, CHENG Zilong, YU Qifeng. Transformer based monocular satellite pose estimation [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(5): 325298-325298.
[7]	WANG Hui, JIA Zikai, JIN Ren, LIN Defu, FAN Junfang, XU Chao. Cooperative object detection in UAV-based vision-guided docking [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(1): 324854-324854.
[8]	LI Hongguang, YU Ruonan, DING Wenrui. Research development of small object traching based on deep learning [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2021, 42(7): 24691-024691.
[9]	JIANG Bo, QU Ruokun, LI Yandong, LI Chenglong. Object detection in UAV imagery based on deep learning: Review [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2021, 42(4): 524519-524519.
[10]	SONG Chuang, ZHAO Jiajia, WANG Kang, LIANG Xinkai. A survey of few shot learning based on intelligent perception [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2020, 41(S1): 723756-723756.
[11]	CHEN Lu, HUANG Panfeng, CAI Jia. Space non-cooperative target detection based on improved features of histogram of oriented gradient [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2016, 37(2): 717-726.
[12]	HAN Yumeng, JIA Xiaohong, LIANG Xiaogeng. Target acquisition probability of infrared imaging missile [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2016, 37(10): 3101-3109.
[13]	LU Yuping, YANG Chaoxing, LIU Yangyang. A Survey of Modeling and Control Technologies for Aerial Refueling System [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2014, 35(9): 2375-2389.
[14]	SHEN Hao, LI Shuxiao, SHEN Yiping, ZHU Chengfei, CHANG Hongxing. Fast Interframe Registration Method in Aerial Videos [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2013, 34(6): 1405-1413.
[15]	Jiang Hongxiang;Xu Jinfa;Gao Zheng. Vision-based Movement State Estimation Algorithm for Unmanned Helicopter Landing [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2010, 31(4): 744-753.

Object detection in UAV images based on multi-scale split attention

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 24

Related Articles 15

Recommended Articles

Metrics

Comments