基于特征增强和校准的航拍车辆实例分割方法

doi:10.7527/S1000-6893.2023.28397

Abstract

CLC Number:

V279

Yucheng YAO, Xu LI, Qimin XU, Dong KONG. Instance segmentation for vehicle in UAV aerial images based on feature enhancement and calibration[J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(24): 328397-328397.

Figures/Tables 14

Fig.1

Fig.2

Fig.3

Fig.4

Fig. 5

Fig.6

Fig.7

Table 1

Comparisons based on different representative networks

基线网络	+本文模型	Params/M	FLOPs/G	$A P m a s k$	$A P S m a s k$	$A P M m a s k$	$A P L m a s k$	$A P b b o x$	$A P S b b o x$	$A P M b b o x$	$A P L b b o x$
YOLACT	×	34.73	186.27	57.1	4.9	43.7	67.0	61.1	13.3	49.2	70.0
YOLACT	√	49.86	207.65	62.2	5.0	44.5	74.3	62.9	15.3	49.7	72.2
Mask R-CNN	×	43.75	258.14	79.3	29	61.8	89.5	80.6	33.6	64.1	90.2
Mask R-CNN	√	58.8	304.75	82.2	33.9	65.6	91.9	83.7	39.5	68.4	92.5
Cascade Mask R-CNN	×	76.8	388.91	81.5	31.9	64.8	91.5	84.4	37.5	68.9	93.8
Cascade Mask R-CNN	√	91.93	435.52	83.1	35.6	67.5	92.5	85.9	41.3	71.8	94.6

Table 1

Table 2

Comparisons of experimental results with different state-of-the-art methods

模型方法	Params/M	FLOPs/G	$A P m a s k$	$A P S m a s k$	$A P M m a s k$	$A P L m a s k$	$A P b b o x$	$A P S b b o x$	$A P M b b o x$	$A P L b b o x$
SOLO	130.1	365.3	70.4	6.9	38.4	87.5
QueryInst	172.22	464.05	76.1	21.6	50.0	90.4	76.1	28.7	51.2	88.8
PointRend	55.53	209.3	80.7	29.6	63.4	90.9	80.7	34.6	65.1	90.1
MSOA-Net	63.55	357.75	81.4	31.7	65.5	90.6	83.2	37.5	68.4	92.3
DetectoRS	134.3	437.35	81.7	32.6	65.1	91.4	84.0	38.1	68.8	93.0
本文方法	58.8	304.7	82.2	33.9	65.6	91.9	83.7	39.5	68.4	92.5

Table 2

Table 3

Ablation studies of each component in our methods

+MSEM	+GLFCM	Transformer类型	Params/M	FLOPs/G	$A P m a s k$	$A P S m a s k$	$A P M m a s k$	$A P L m a s k$	$A P b b o x$	$A P S b b o x$	$A P M b b o x$	$A P L b b o x$
×	×	×	43.75	258.1	79.3	29.0	61.8	89.5	80.6	33.6	64.1	90.2
√	×	×	56.7	271.1	80.1	30.2	63.0	90.0	81.5	35.0	65.7	90.6
×	√	原始	46.11	459.4	81.6	32.9	64.5	91.3	82.8	38.3	67.3	92.0
×	√	线性	45.92	291.7	81.5	32.7	64.8	91.3	83.0	37.4	67.4	92.1
√	√	原始	60.7	577.3	82.2	34.3	65.5	91.9	83.4	39.8	68.6	92.2
√	√	线性	58.8	304.7	82.2	33.9	65.6	91.9	83.7	39.5	68.4	92.5

Table 3

Table4

Ablation studies of each component in MSEM

+空洞卷积模块	+空间注意力模块	Params/M	FLOPs/G	$A P m a s k$	$A P S m a s k$	$A P M m a s k$	$A P L m a s k$	$A P b b o x$	$A P S b b o x$	$A P M b b o x$	$A P L b b o x$
×	×	43.7	258.1	79.3	29.0	61.8	89.5	80.6	33.6	64.1	90.2
×	√	43.9	258.3	79.4	28.9	61.9	89.6	80.6	33.4	64.2	90.2
√	×	56.5	270.9	79.8	30.0	62.5	89.7	81.3	34.8	65.3	90.5
√	√	56.7	271.1	80.1	30.2	63.0	90.0	81.5	35.0	65.7	90.6

Table4

Fig.8

Fig.9

Table A1

Comparison of enhancement effect of different features on network performance by MSEM

$F 5$	$F 4$	$F 3$	Params/M	FLOPs/G	$A P m a s k$	$A P S m a s k$	$A P M m a s k$	$A P L m a s k$	$A P b b o x$	$A P S b b o x$	$A P M b b o x$	$A P L b b o x$
√			56.7	271.1	80.1	30.2	63.0	90.0	81.5	35.0	65.7	90.6
√	√		67.1	312.5	80.0	30.4	62.8	89.8	81.5	34.7	65.7	90.8
√	√	√	76.1	457.4	80.2	30.5	63.0	89.9	81.6	35.1	65.8	90.8

Table A1

References 32

1	OSCO L P， MARCATO J Jr， MARQUES RAMOS A P， et al. A review on deep learning in UAV remote sensing［J］. International Journal of Applied Earth Observation and Geoinformation， 2021， 102： 102456.
2	ZHANG H， SUN H， AO W， et al. A survey on instance segmentation： Recent advances and challenges［J］. International Journal of Innovative Computing， Information and Control， 2021， 17（3）： 1041-1053.
3	MOU L C， ZHU X X. Vehicle instance segmentation from aerial image and video using a multitask learning residual fully convolutional network［J］. IEEE Transactions on Geoscience and Remote Sensing， 2018， 56（11）： 6699-6711.
4	AMMOUR N， ALHICHRI H， BAZI Y， et al. Deep learning approach for car detection in UAV imagery［J］. Remote Sensing， 2017， 9（4）： 312.
5	SIMONYAN K， ZISSERMAN A. Very deep convolutional networks for large-scale image recognition［DB/OL］. arXiv preprint： 1409.1556， 2014.
6	AUDEBERT N， LE SAUX B， LEFÈVRE S. Segment-before-detect： Vehicle detection and classification through semantic segmentation of aerial images［J］. Remote Sensing， 2017， 9（4）： 368.
7	BADRINARAYANAN V， KENDALL A， CIPOLLA R. SegNet： A deep convolutional encoder-decoder architecture for image segmentation［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（12）： 2481-2495.
8	BOLYA D， ZHOU C， XIAO F Y， et al. YOLACT： Real-time instance segmentation［C］∥2019 IEEE/CVF International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2020： 9156-9165.
9	TIAN Z， SHEN C H， CHEN H. Conditional convolutions for instance segmentation［M］∥Computer Vision–ECCV 2020. Cham： Springer International Publishing， 2020： 282-298.
10	SU H， HUANG P， YIN J， et al. Faster and better instance segmentation for large scene remote sensing imagery［C］∥IGARSS 2022 - 2022 IEEE International Geoscience and Remote Sensing Symposium. Piscataway： IEEE Press， 2022： 2187-2190.
11	YU F， WANG D Q， SHELHAMER E， et al. Deep layer aggregation［C］∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2018： 2403-2412.
12	TAN M X， PANG R M， LE Q V. EfficientDet： Scalable and efficient object detection［C］∥2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2020： 10778-10787.
13	HE K M， GKIOXARI G， DOLLÁR P， et al. Mask R-CNN［C］∥2017 IEEE International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2017： 2980-2988.
14	WANG J W， DING J A， GUO H W， et al. Mask OBB： A semantic attention-based mask oriented bounding box representation for multi-category object detection in aerial images［J］. Remote Sensing， 2019， 11（24）： 2930.
15	PAN T， DING J， WANG J W， et al. Instance segmentation with oriented proposals for aerial images［C］∥IGARSS 2020-2020 IEEE International Geoscience and Remote Sensing Symposium. Piscataway： IEEE Press， 2021： 988-991.
16	HU J， CAO L J， LU Y， et al. ISTR： End-to-end instance segmentation with transformers［DB/OL］. arXiv preprint： 2105.00637， 2021.
17	ZHANG W， LIU C S， CHANG F L， et al. Multi-scale and occlusion aware network for vehicle detection and segmentation on UAV aerial images［J］. Remote Sensing， 2020， 12（11）： 1760.
18	CAI Z W， VASCONCELOS N. Cascade R-CNN： High quality object detection and instance segmentation［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2021， 43（5）： 1483-1498.
19	CHEN K， PANG J M， WANG J Q， et al. Hybrid task cascade for instance segmentation［C］∥2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2020： 4969-4978.
20	SU H， WEI S J， LIU S， et al. HQ-ISNet： High-quality instance segmentation for remote sensing imagery［J］. Remote Sensing， 2020， 12（6）： 989.
21	VASWANI A， BRAIN G， SHAZEER N， et al. Attention is all you need［C］∥Advances in Neural Information Processing Systems， 2017： 30.
22	QIN Z， LI Z M， ZHANG Z N， et al. ThunderNet： Towards real-time generic object detection on mobile devices［C］∥2019 IEEE/CVF International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2020： 6717-6726.
23	YANG M K， YU K， ZHANG C， et al. DenseASPP for semantic segmentation in street scenes［C］∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2018： 3684-3692.
24	WANG X L， GIRSHICK R， GUPTA A， et al. Non-local neural networks［C］∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2018： 7794-7803.
25	PECORARO R， BASILE V， BONO V. Local multi-head channel self-attention for facial expression recognition［J］. Information， 2022， 13（9）： 419.
26	TSAI Y H H， BAI S J， YAMADA M， et al. Transformer dissection： A unified understanding of transformer’s attention via the lens of kernel［DB/OL］. arXiv preprint： 1908.11775， 2019.
27	KATHAROPOULOS A， VYAS A， PAPPAS N， et al. Transformers are RNNs： Fast autoregressive transformers with linear attention［DB/OL］. arXiv preprint： 2006.16236， 2020.
28	QIN Z， SUN W X， DENG H， et al. cosFormer： Rethinking softmax in attention［DB/OL］. arXiv preprint： 2202.08791， 2022.
29	WANG X L， KONG T， SHEN C H， et al. SOLO： Segmenting objects by locations［M］∥Computer Vision- ECCV 2020. Cham： Springer International Publishing， 2020： 649-665.
30	FANG Y X， YANG S S， WANG X G， et al. Instances as queries［C］∥2021 IEEE/CVF International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2022： 6890-6899.
31	KIRILLOV A， WU Y X， HE K M， et al. PointRend： Image segmentation As rendering［C］∥2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2020： 9796-9805.
32	QIAO S Y， CHEN L C， YUILLE A. DetectoRS： Detecting objects with recursive feature pyramid and switchable atrous convolution［C］∥2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2021： 10208-10219.

[1]	. Automatic Landing Method for Quad-rotor Helicopter Based on MDP [J]. Acta Aeronautica et Astronautica Sinica, 0, (): 0-0.
[2]	. Trajectory Tracking Control Algorithm for Canard-equipped Tail-Sitting Vertical Takeoff and Landing UAV based on MPCC [J]. Acta Aeronautica et Astronautica Sinica, 0, (): 0-0.
[3]	. Vision-based air-to-air multi-UAVs tracking [J]. Acta Aeronautica et Astronautica Sinica, 0, (): 0-0.
[4]	. Coordinated Manned/Unmanned Aerial Warfare: Concepts, Technologies, and Challenges [J]. Acta Aeronautica et Astronautica Sinica, 0, (): 0-0.
[5]	Wenjun DING, Yajun CHAI, Dongdong HOU, Chiyu WANG, Guozong ZHANG, Zhaoyong MAO. Path planning for AUV&UAV cross⁃domain collaborative search and tracking [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(21): 528471-528471.
[6]	Ao WU, Huanyu LI, Yangjun GAO, Longfei YUE, Yajie CUI, Rennong YANG. Time-space coordination method for air refueling management for combat air patrol position maintenance [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(20): 628438-628438.
[7]	Tongyan WU, Mengzhen HUO, Haibin DUAN, Yimin DENG. Biologically eagle-eye and midbrain mechanism-based saliency detection of UAV aerial refueling targets [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(20): 628492-628492.
[8]	Yubin YUAN, Yiquan WU, Langyue ZHAO, Jinlin CHEN, Qichang ZHAO. Research progress of UAV aerial video multi⁃object detection and tracking based on deep learning [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(18): 28334-028334.
[9]	Shuyi GAO, Defu LIN, Duo ZHENG, Xinyu HU. Intelligent cooperative interception strategy of aircraft against cluster attack [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(18): 328301-328301.
[10]	. The Application of Laser Weapon Anti-UAV Swarm and Key Technical Research [J]. Acta Aeronautica et Astronautica Sinica, 0, (): 0-0.
[11]	Yougang XIAO, Xiangna MAN, Guohua WU, Qizhang LUO. Surveying and mapping path planning method for UAV-borne SAR considering terains [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(17): 328143-328143.
[12]	Meng-Tong ZHANG Guangtong Xu. Multi-UAV Cooperative Path Planning Based on Multi-index Dynamic Priority [J]. Acta Aeronautica et Astronautica Sinica, 0, (): 0-0.
[13]	. Small Target Detection Algorithm for UAV Based on Patch-Wise Co-Attention [J]. Acta Aeronautica et Astronautica Sinica, 0, (): 0-0.
[14]	Yafei LU, Qingyang CHEN, Peng WANG, Bo WANG, Zheng GUO. Design and experiment of a small air-launched UAV [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(15): 528642-528642.
[15]	Bo SHEN, Wenliang WU, Gang YANG, Xingshe ZHOU. Evaluation models and methods for intelligence of unmanned swarm systems based on collective OODA loop [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(14): 328003-328003.

Instance segmentation for vehicle in UAV aerial images based on feature enhancement and calibration

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 14

References 32

Related Articles 15

Recommended Articles

Metrics

Comments