基于特征增强和校准的航拍车辆实例分割方法

doi:10.7527/S1000-6893.2023.28397

Abstract

CLC Number:

V279

Yucheng YAO, Xu LI, Qimin XU, Dong KONG. Instance segmentation for vehicle in UAV aerial images based on feature enhancement and calibration[J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(24): 328397.

Figures/Tables 14

Fig.1

Fig.2

Fig.3

Fig.4

Fig. 5

Fig.6

Fig.7

Table 1

Comparisons based on different representative networks

基线网络	+本文模型	Params/M	FLOPs/G	$A P m a s k$	$A P S m a s k$	$A P M m a s k$	$A P L m a s k$	$A P b b o x$	$A P S b b o x$	$A P M b b o x$	$A P L b b o x$
YOLACT	×	34.73	186.27	57.1	4.9	43.7	67.0	61.1	13.3	49.2	70.0
YOLACT	√	49.86	207.65	62.2	5.0	44.5	74.3	62.9	15.3	49.7	72.2
Mask R-CNN	×	43.75	258.14	79.3	29	61.8	89.5	80.6	33.6	64.1	90.2
Mask R-CNN	√	58.8	304.75	82.2	33.9	65.6	91.9	83.7	39.5	68.4	92.5
Cascade Mask R-CNN	×	76.8	388.91	81.5	31.9	64.8	91.5	84.4	37.5	68.9	93.8
Cascade Mask R-CNN	√	91.93	435.52	83.1	35.6	67.5	92.5	85.9	41.3	71.8	94.6

Table 1

Table 2

Comparisons of experimental results with different state-of-the-art methods

模型方法	Params/M	FLOPs/G	$A P m a s k$	$A P S m a s k$	$A P M m a s k$	$A P L m a s k$	$A P b b o x$	$A P S b b o x$	$A P M b b o x$	$A P L b b o x$
SOLO	130.1	365.3	70.4	6.9	38.4	87.5
QueryInst	172.22	464.05	76.1	21.6	50.0	90.4	76.1	28.7	51.2	88.8
PointRend	55.53	209.3	80.7	29.6	63.4	90.9	80.7	34.6	65.1	90.1
MSOA-Net	63.55	357.75	81.4	31.7	65.5	90.6	83.2	37.5	68.4	92.3
DetectoRS	134.3	437.35	81.7	32.6	65.1	91.4	84.0	38.1	68.8	93.0
本文方法	58.8	304.7	82.2	33.9	65.6	91.9	83.7	39.5	68.4	92.5

Table 2

Table 3

Ablation studies of each component in our methods

+MSEM	+GLFCM	Transformer类型	Params/M	FLOPs/G	$A P m a s k$	$A P S m a s k$	$A P M m a s k$	$A P L m a s k$	$A P b b o x$	$A P S b b o x$	$A P M b b o x$	$A P L b b o x$
×	×	×	43.75	258.1	79.3	29.0	61.8	89.5	80.6	33.6	64.1	90.2
√	×	×	56.7	271.1	80.1	30.2	63.0	90.0	81.5	35.0	65.7	90.6
×	√	原始	46.11	459.4	81.6	32.9	64.5	91.3	82.8	38.3	67.3	92.0
×	√	线性	45.92	291.7	81.5	32.7	64.8	91.3	83.0	37.4	67.4	92.1
√	√	原始	60.7	577.3	82.2	34.3	65.5	91.9	83.4	39.8	68.6	92.2
√	√	线性	58.8	304.7	82.2	33.9	65.6	91.9	83.7	39.5	68.4	92.5

Table 3

Table4

Ablation studies of each component in MSEM

+空洞卷积模块	+空间注意力模块	Params/M	FLOPs/G	$A P m a s k$	$A P S m a s k$	$A P M m a s k$	$A P L m a s k$	$A P b b o x$	$A P S b b o x$	$A P M b b o x$	$A P L b b o x$
×	×	43.7	258.1	79.3	29.0	61.8	89.5	80.6	33.6	64.1	90.2
×	√	43.9	258.3	79.4	28.9	61.9	89.6	80.6	33.4	64.2	90.2
√	×	56.5	270.9	79.8	30.0	62.5	89.7	81.3	34.8	65.3	90.5
√	√	56.7	271.1	80.1	30.2	63.0	90.0	81.5	35.0	65.7	90.6

Table4

Fig.8

Fig.9

Table A1

Comparison of enhancement effect of different features on network performance by MSEM

$F 5$	$F 4$	$F 3$	Params/M	FLOPs/G	$A P m a s k$	$A P S m a s k$	$A P M m a s k$	$A P L m a s k$	$A P b b o x$	$A P S b b o x$	$A P M b b o x$	$A P L b b o x$
√			56.7	271.1	80.1	30.2	63.0	90.0	81.5	35.0	65.7	90.6
√	√		67.1	312.5	80.0	30.4	62.8	89.8	81.5	34.7	65.7	90.8
√	√	√	76.1	457.4	80.2	30.5	63.0	89.9	81.6	35.1	65.8	90.8

Table A1

References 32

1	OSCO L P， MARCATO J Jr， MARQUES RAMOS A P， et al. A review on deep learning in UAV remote sensing［J］. International Journal of Applied Earth Observation and Geoinformation， 2021， 102： 102456.
2	ZHANG H， SUN H， AO W， et al. A survey on instance segmentation： Recent advances and challenges［J］. International Journal of Innovative Computing， Information and Control， 2021， 17（3）： 1041-1053.
3	MOU L C， ZHU X X. Vehicle instance segmentation from aerial image and video using a multitask learning residual fully convolutional network［J］. IEEE Transactions on Geoscience and Remote Sensing， 2018， 56（11）： 6699-6711.
4	AMMOUR N， ALHICHRI H， BAZI Y， et al. Deep learning approach for car detection in UAV imagery［J］. Remote Sensing， 2017， 9（4）： 312.
5	SIMONYAN K， ZISSERMAN A. Very deep convolutional networks for large-scale image recognition［DB/OL］. arXiv preprint： 1409.1556， 2014.
6	AUDEBERT N， LE SAUX B， LEFÈVRE S. Segment-before-detect： Vehicle detection and classification through semantic segmentation of aerial images［J］. Remote Sensing， 2017， 9（4）： 368.
7	BADRINARAYANAN V， KENDALL A， CIPOLLA R. SegNet： A deep convolutional encoder-decoder architecture for image segmentation［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（12）： 2481-2495.
8	BOLYA D， ZHOU C， XIAO F Y， et al. YOLACT： Real-time instance segmentation［C］∥2019 IEEE/CVF International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2020： 9156-9165.
9	TIAN Z， SHEN C H， CHEN H. Conditional convolutions for instance segmentation［M］∥Computer Vision–ECCV 2020. Cham： Springer International Publishing， 2020： 282-298.
10	SU H， HUANG P， YIN J， et al. Faster and better instance segmentation for large scene remote sensing imagery［C］∥IGARSS 2022 - 2022 IEEE International Geoscience and Remote Sensing Symposium. Piscataway： IEEE Press， 2022： 2187-2190.
11	YU F， WANG D Q， SHELHAMER E， et al. Deep layer aggregation［C］∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2018： 2403-2412.
12	TAN M X， PANG R M， LE Q V. EfficientDet： Scalable and efficient object detection［C］∥2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2020： 10778-10787.
13	HE K M， GKIOXARI G， DOLLÁR P， et al. Mask R-CNN［C］∥2017 IEEE International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2017： 2980-2988.
14	WANG J W， DING J A， GUO H W， et al. Mask OBB： A semantic attention-based mask oriented bounding box representation for multi-category object detection in aerial images［J］. Remote Sensing， 2019， 11（24）： 2930.
15	PAN T， DING J， WANG J W， et al. Instance segmentation with oriented proposals for aerial images［C］∥IGARSS 2020-2020 IEEE International Geoscience and Remote Sensing Symposium. Piscataway： IEEE Press， 2021： 988-991.
16	HU J， CAO L J， LU Y， et al. ISTR： End-to-end instance segmentation with transformers［DB/OL］. arXiv preprint： 2105.00637， 2021.
17	ZHANG W， LIU C S， CHANG F L， et al. Multi-scale and occlusion aware network for vehicle detection and segmentation on UAV aerial images［J］. Remote Sensing， 2020， 12（11）： 1760.
18	CAI Z W， VASCONCELOS N. Cascade R-CNN： High quality object detection and instance segmentation［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2021， 43（5）： 1483-1498.
19	CHEN K， PANG J M， WANG J Q， et al. Hybrid task cascade for instance segmentation［C］∥2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2020： 4969-4978.
20	SU H， WEI S J， LIU S， et al. HQ-ISNet： High-quality instance segmentation for remote sensing imagery［J］. Remote Sensing， 2020， 12（6）： 989.
21	VASWANI A， BRAIN G， SHAZEER N， et al. Attention is all you need［C］∥Advances in Neural Information Processing Systems， 2017： 30.
22	QIN Z， LI Z M， ZHANG Z N， et al. ThunderNet： Towards real-time generic object detection on mobile devices［C］∥2019 IEEE/CVF International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2020： 6717-6726.
23	YANG M K， YU K， ZHANG C， et al. DenseASPP for semantic segmentation in street scenes［C］∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2018： 3684-3692.
24	WANG X L， GIRSHICK R， GUPTA A， et al. Non-local neural networks［C］∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2018： 7794-7803.
25	PECORARO R， BASILE V， BONO V. Local multi-head channel self-attention for facial expression recognition［J］. Information， 2022， 13（9）： 419.
26	TSAI Y H H， BAI S J， YAMADA M， et al. Transformer dissection： A unified understanding of transformer’s attention via the lens of kernel［DB/OL］. arXiv preprint： 1908.11775， 2019.
27	KATHAROPOULOS A， VYAS A， PAPPAS N， et al. Transformers are RNNs： Fast autoregressive transformers with linear attention［DB/OL］. arXiv preprint： 2006.16236， 2020.
28	QIN Z， SUN W X， DENG H， et al. cosFormer： Rethinking softmax in attention［DB/OL］. arXiv preprint： 2202.08791， 2022.
29	WANG X L， KONG T， SHEN C H， et al. SOLO： Segmenting objects by locations［M］∥Computer Vision- ECCV 2020. Cham： Springer International Publishing， 2020： 649-665.
30	FANG Y X， YANG S S， WANG X G， et al. Instances as queries［C］∥2021 IEEE/CVF International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2022： 6890-6899.
31	KIRILLOV A， WU Y X， HE K M， et al. PointRend： Image segmentation As rendering［C］∥2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2020： 9796-9805.
32	QIAO S Y， CHEN L C， YUILLE A. DetectoRS： Detecting objects with recursive feature pyramid and switchable atrous convolution［C］∥2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2021： 10208-10219.

[1]	Chen WANG, Caisheng WEI, Zeyang YIN, Kai JIN, Xingchen LI. Collaborative planning of multi-UAV trajectories and communication strategies considering channel resource constraints [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(18): 331837-331837.
[2]	Yi ZHENG, Xianghong CHENG, Xingbang TANG, Yi CAO. Oriented detection algorithm for insulator and their defects from aerial images based on improved ReDet [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(18): 331825-331825.
[3]	. Robust infrared target tracking algorithm for anti-UAV in complex backgrounds [J]. Acta Aeronautica et Astronautica Sinica, 0, (): 1-0.
[4]	. Research on UAV Complete Data Collection Trajectory Planning Algorithm Based on Time Window Constraints [J]. Acta Aeronautica et Astronautica Sinica, 0, (): 1-0.
[5]	Fang LIU, Chenyang LU, Yan LU, Xin WANG. Adaptive template update-based Transformer algorithm for UAV target tracking [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(16): 331687-331687.
[6]	Yonggang YANG, Wentao JIANG, Zhiyun GAO. Real-time target detection algorithm for low altitude UAVs [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(16): 331619-331619.
[7]	Yu WANG, Zhipeng XIE, Yongjian TIAN, Guanglei MENG. Distributed UAV formation control with virtual structure guided reinforcement learning [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(15): 331354-331354.
[8]	. Risk-aware autonomous avoidance for eVTOL [J]. Acta Aeronautica et Astronautica Sinica, 0, (): 1-0.
[9]	Fanteng MENG, Yong QIN, Jing CUI, Yunpeng WU, Zicheng ZHANG, Shaowei WEI. Unknown risk detection in external environment of railroad using UAV images [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(11): 531262-531262.
[10]	Lin CHEN, Qing ZHU, Han HU, Yulin DING, Pengxin GU. FLASH: Flexible and lightweight awareness of slope hazard [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(11): 531293-531293.
[11]	Junzhi LI, Teng LONG, Jingliang SUN, Hongyu MIAO, Zhenlin ZHOU. Differential flatness-based spatial-temporal hierarchical trajectory planning for fixed-wing UAVs in urban environments [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(11): 531369-531369.
[12]	Yongguang JIN, Fangwei YE, Qihui WU. Location privacy protection mechanisms for UAVs with Remote ID [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(11): 531341-531341.
[13]	Ruokun QU, Zhiyuan WANG, Yelu LIU, Chenglong LI, Bo JIANG. UAV visual positioning technology for urban air mobility [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(11): 531168-531168.
[14]	Haijun ZHANG, Qingyue XIA, Xu MA, Chao REN, Yang LU. A review of unmanned aerial vehicles deployment optimization in 6G low-altitude communication scenarios [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(11): 531296-531296.
[15]	. RGB-T UAV Object Tracking Based on Feature-cooperative Reconstruction [J]. Acta Aeronautica et Astronautica Sinica, 0, (): 1-0.

Instance segmentation for vehicle in UAV aerial images based on feature enhancement and calibration

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 14

References 32

Related Articles 15

Recommended Articles

Metrics

Comments