基于特征增强和校准的航拍车辆实例分割方法

doi:10.7527/S1000-6893.2023.28397

Abstract

CLC Number:

V279

Yucheng YAO, Xu LI, Qimin XU, Dong KONG. Instance segmentation for vehicle in UAV aerial images based on feature enhancement and calibration[J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(24): 328397-328397.

Figures/Tables 14

Fig.1

Fig.2

Fig.3

Fig.4

Fig. 5

Fig.6

Fig.7

Table 1

Comparisons based on different representative networks

基线网络	+本文模型	Params/M	FLOPs/G	$A P m a s k$	$A P S m a s k$	$A P M m a s k$	$A P L m a s k$	$A P b b o x$	$A P S b b o x$	$A P M b b o x$	$A P L b b o x$
YOLACT	×	34.73	186.27	57.1	4.9	43.7	67.0	61.1	13.3	49.2	70.0
YOLACT	√	49.86	207.65	62.2	5.0	44.5	74.3	62.9	15.3	49.7	72.2
Mask R-CNN	×	43.75	258.14	79.3	29	61.8	89.5	80.6	33.6	64.1	90.2
Mask R-CNN	√	58.8	304.75	82.2	33.9	65.6	91.9	83.7	39.5	68.4	92.5
Cascade Mask R-CNN	×	76.8	388.91	81.5	31.9	64.8	91.5	84.4	37.5	68.9	93.8
Cascade Mask R-CNN	√	91.93	435.52	83.1	35.6	67.5	92.5	85.9	41.3	71.8	94.6

Table 1

Table 2

Comparisons of experimental results with different state-of-the-art methods

模型方法	Params/M	FLOPs/G	$A P m a s k$	$A P S m a s k$	$A P M m a s k$	$A P L m a s k$	$A P b b o x$	$A P S b b o x$	$A P M b b o x$	$A P L b b o x$
SOLO	130.1	365.3	70.4	6.9	38.4	87.5
QueryInst	172.22	464.05	76.1	21.6	50.0	90.4	76.1	28.7	51.2	88.8
PointRend	55.53	209.3	80.7	29.6	63.4	90.9	80.7	34.6	65.1	90.1
MSOA-Net	63.55	357.75	81.4	31.7	65.5	90.6	83.2	37.5	68.4	92.3
DetectoRS	134.3	437.35	81.7	32.6	65.1	91.4	84.0	38.1	68.8	93.0
本文方法	58.8	304.7	82.2	33.9	65.6	91.9	83.7	39.5	68.4	92.5

Table 2

Table 3

Ablation studies of each component in our methods

+MSEM	+GLFCM	Transformer类型	Params/M	FLOPs/G	$A P m a s k$	$A P S m a s k$	$A P M m a s k$	$A P L m a s k$	$A P b b o x$	$A P S b b o x$	$A P M b b o x$	$A P L b b o x$
×	×	×	43.75	258.1	79.3	29.0	61.8	89.5	80.6	33.6	64.1	90.2
√	×	×	56.7	271.1	80.1	30.2	63.0	90.0	81.5	35.0	65.7	90.6
×	√	原始	46.11	459.4	81.6	32.9	64.5	91.3	82.8	38.3	67.3	92.0
×	√	线性	45.92	291.7	81.5	32.7	64.8	91.3	83.0	37.4	67.4	92.1
√	√	原始	60.7	577.3	82.2	34.3	65.5	91.9	83.4	39.8	68.6	92.2
√	√	线性	58.8	304.7	82.2	33.9	65.6	91.9	83.7	39.5	68.4	92.5

Table 3

Table4

Ablation studies of each component in MSEM

+空洞卷积模块	+空间注意力模块	Params/M	FLOPs/G	$A P m a s k$	$A P S m a s k$	$A P M m a s k$	$A P L m a s k$	$A P b b o x$	$A P S b b o x$	$A P M b b o x$	$A P L b b o x$
×	×	43.7	258.1	79.3	29.0	61.8	89.5	80.6	33.6	64.1	90.2
×	√	43.9	258.3	79.4	28.9	61.9	89.6	80.6	33.4	64.2	90.2
√	×	56.5	270.9	79.8	30.0	62.5	89.7	81.3	34.8	65.3	90.5
√	√	56.7	271.1	80.1	30.2	63.0	90.0	81.5	35.0	65.7	90.6

Table4

Fig.8

Fig.9

Table A1

Comparison of enhancement effect of different features on network performance by MSEM

$F 5$	$F 4$	$F 3$	Params/M	FLOPs/G	$A P m a s k$	$A P S m a s k$	$A P M m a s k$	$A P L m a s k$	$A P b b o x$	$A P S b b o x$	$A P M b b o x$	$A P L b b o x$
√			56.7	271.1	80.1	30.2	63.0	90.0	81.5	35.0	65.7	90.6
√	√		67.1	312.5	80.0	30.4	62.8	89.8	81.5	34.7	65.7	90.8
√	√	√	76.1	457.4	80.2	30.5	63.0	89.9	81.6	35.1	65.8	90.8

Table A1

References 32

1	OSCO L P， MARCATO J Jr， MARQUES RAMOS A P， et al. A review on deep learning in UAV remote sensing［J］. International Journal of Applied Earth Observation and Geoinformation， 2021， 102： 102456.
2	ZHANG H， SUN H， AO W， et al. A survey on instance segmentation： Recent advances and challenges［J］. International Journal of Innovative Computing， Information and Control， 2021， 17（3）： 1041-1053.
3	MOU L C， ZHU X X. Vehicle instance segmentation from aerial image and video using a multitask learning residual fully convolutional network［J］. IEEE Transactions on Geoscience and Remote Sensing， 2018， 56（11）： 6699-6711.
4	AMMOUR N， ALHICHRI H， BAZI Y， et al. Deep learning approach for car detection in UAV imagery［J］. Remote Sensing， 2017， 9（4）： 312.
5	SIMONYAN K， ZISSERMAN A. Very deep convolutional networks for large-scale image recognition［DB/OL］. arXiv preprint： 1409.1556， 2014.
6	AUDEBERT N， LE SAUX B， LEFÈVRE S. Segment-before-detect： Vehicle detection and classification through semantic segmentation of aerial images［J］. Remote Sensing， 2017， 9（4）： 368.
7	BADRINARAYANAN V， KENDALL A， CIPOLLA R. SegNet： A deep convolutional encoder-decoder architecture for image segmentation［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（12）： 2481-2495.
8	BOLYA D， ZHOU C， XIAO F Y， et al. YOLACT： Real-time instance segmentation［C］∥2019 IEEE/CVF International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2020： 9156-9165.
9	TIAN Z， SHEN C H， CHEN H. Conditional convolutions for instance segmentation［M］∥Computer Vision–ECCV 2020. Cham： Springer International Publishing， 2020： 282-298.
10	SU H， HUANG P， YIN J， et al. Faster and better instance segmentation for large scene remote sensing imagery［C］∥IGARSS 2022 - 2022 IEEE International Geoscience and Remote Sensing Symposium. Piscataway： IEEE Press， 2022： 2187-2190.
11	YU F， WANG D Q， SHELHAMER E， et al. Deep layer aggregation［C］∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2018： 2403-2412.
12	TAN M X， PANG R M， LE Q V. EfficientDet： Scalable and efficient object detection［C］∥2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2020： 10778-10787.
13	HE K M， GKIOXARI G， DOLLÁR P， et al. Mask R-CNN［C］∥2017 IEEE International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2017： 2980-2988.
14	WANG J W， DING J A， GUO H W， et al. Mask OBB： A semantic attention-based mask oriented bounding box representation for multi-category object detection in aerial images［J］. Remote Sensing， 2019， 11（24）： 2930.
15	PAN T， DING J， WANG J W， et al. Instance segmentation with oriented proposals for aerial images［C］∥IGARSS 2020-2020 IEEE International Geoscience and Remote Sensing Symposium. Piscataway： IEEE Press， 2021： 988-991.
16	HU J， CAO L J， LU Y， et al. ISTR： End-to-end instance segmentation with transformers［DB/OL］. arXiv preprint： 2105.00637， 2021.
17	ZHANG W， LIU C S， CHANG F L， et al. Multi-scale and occlusion aware network for vehicle detection and segmentation on UAV aerial images［J］. Remote Sensing， 2020， 12（11）： 1760.
18	CAI Z W， VASCONCELOS N. Cascade R-CNN： High quality object detection and instance segmentation［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2021， 43（5）： 1483-1498.
19	CHEN K， PANG J M， WANG J Q， et al. Hybrid task cascade for instance segmentation［C］∥2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2020： 4969-4978.
20	SU H， WEI S J， LIU S， et al. HQ-ISNet： High-quality instance segmentation for remote sensing imagery［J］. Remote Sensing， 2020， 12（6）： 989.
21	VASWANI A， BRAIN G， SHAZEER N， et al. Attention is all you need［C］∥Advances in Neural Information Processing Systems， 2017： 30.
22	QIN Z， LI Z M， ZHANG Z N， et al. ThunderNet： Towards real-time generic object detection on mobile devices［C］∥2019 IEEE/CVF International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2020： 6717-6726.
23	YANG M K， YU K， ZHANG C， et al. DenseASPP for semantic segmentation in street scenes［C］∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2018： 3684-3692.
24	WANG X L， GIRSHICK R， GUPTA A， et al. Non-local neural networks［C］∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2018： 7794-7803.
25	PECORARO R， BASILE V， BONO V. Local multi-head channel self-attention for facial expression recognition［J］. Information， 2022， 13（9）： 419.
26	TSAI Y H H， BAI S J， YAMADA M， et al. Transformer dissection： A unified understanding of transformer’s attention via the lens of kernel［DB/OL］. arXiv preprint： 1908.11775， 2019.
27	KATHAROPOULOS A， VYAS A， PAPPAS N， et al. Transformers are RNNs： Fast autoregressive transformers with linear attention［DB/OL］. arXiv preprint： 2006.16236， 2020.
28	QIN Z， SUN W X， DENG H， et al. cosFormer： Rethinking softmax in attention［DB/OL］. arXiv preprint： 2202.08791， 2022.
29	WANG X L， KONG T， SHEN C H， et al. SOLO： Segmenting objects by locations［M］∥Computer Vision- ECCV 2020. Cham： Springer International Publishing， 2020： 649-665.
30	FANG Y X， YANG S S， WANG X G， et al. Instances as queries［C］∥2021 IEEE/CVF International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2022： 6890-6899.
31	KIRILLOV A， WU Y X， HE K M， et al. PointRend： Image segmentation As rendering［C］∥2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2020： 9796-9805.
32	QIAO S Y， CHEN L C， YUILLE A. DetectoRS： Detecting objects with recursive feature pyramid and switchable atrous convolution［C］∥2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2021： 10208-10219.

[1]	Chuanyun WANG, Yang SU, Linlin WANG, Tian WANG, Jingjing WANG, Qian GAO. Multi-object continuous robust tracking algorithm for anti-UAV swarm [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(7): 329017-329017.
[2]	Xudong LUO, Yiquan WU, Jinlin CHEN. Research progress on deep learning methods for object detection and semantic segmentation in UAV aerial images [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(6): 28822-028822.
[3]	Hongyu YIN, Yu WU, Tianjiao LIANG. Cooperative path planning for patrol coverage of fixed wing UAV [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(6): 328944-328944.
[4]	. Integrated design of an high altitude super long endurance UAV system driven by distributed ground microwave power [J]. Acta Aeronautica et Astronautica Sinica, 0, (): 0-0.
[5]	Xin-Yu XU Jian Chen. UAV object tracking for air-ground targets based on status detection and Kalman filter [J]. Acta Aeronautica et Astronautica Sinica, 0, (): 0-0.
[6]	Zhu WANG, Mengtong ZHANG, Zhenpeng ZHANG, Guangtong XU. Multi-UAV cooperative path planning based on multi-index dynamic priority [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(4): 328816-328816.
[7]	Haiqiao LIU, Meng LIU, Zichao GONG, Jing DONG. A review of image matching methods based on deep learning [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(3): 28796-028796.
[8]	. Directional Expelling Attack for Unmanned Aerial Vehicle Swarm with leader-following structure [J]. Acta Aeronautica et Astronautica Sinica, 0, (): 0-0.
[9]	. Cooperative Guidance Technique Considering Flight Safety Constraints of Cluster Wake Vortex Aerodynamic Coupling Effects [J]. Acta Aeronautica et Astronautica Sinica, 0, (): 0-0.
[10]	. Automatic Landing Method for Quad-rotor Helicopter Based on MDP [J]. Acta Aeronautica et Astronautica Sinica, 0, (): 0-0.
[11]	Yuqi CAO, Haoran FU, Fei GAO, Ximin LYU. Trajectory tracking control algorithm for canard⁃equipped tail⁃sitting vertical takeoff and landing UAV based on MPCC [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(S2): 729950-729950.
[12]	. Coordinated Manned/Unmanned Aerial Warfare: Concepts, Technologies, and Challenges [J]. Acta Aeronautica et Astronautica Sinica, 0, (): 0-0.
[13]	. Vision-based air-to-air multi-UAVs tracking [J]. Acta Aeronautica et Astronautica Sinica, 0, (): 0-0.
[14]	Wenjun DING, Yajun CHAI, Dongdong HOU, Chiyu WANG, Guozong ZHANG, Zhaoyong MAO. Path planning for AUV&UAV cross⁃domain collaborative search and tracking [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(21): 528471-528471.
[15]	Tongyan WU, Mengzhen HUO, Haibin DUAN, Yimin DENG. Biologically eagle-eye and midbrain mechanism-based saliency detection of UAV aerial refueling targets [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(20): 628492-628492.

Instance segmentation for vehicle in UAV aerial images based on feature enhancement and calibration

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 14

References 32

Related Articles 15

Recommended Articles

Metrics

Comments