基于多模型协同的舰船目标检测

doi:10.7527/S1000-6893.2024.30241

Abstract

Abstract:

In the context of globalization， the importance of ship monitoring is becoming more and more prominent. With the continuous progress of the remote sensing imaging technology， ship detection has become a key means to ensure the safety and efficiency of maritime transportation， and is crucial for maritime transportation， environmental protection， and national security. However， due to the large difference in scales and complex background of ship targets， existing single detection model methods rely too heavily on training data and cannot adapt to ship targets with variable scales. In this paper， we propose a multiple models collaboration framework， in which multiple trained ship detection models are regarded as auxiliary network， and the main network training is optimized by knowledge migration. First， ternary relationship constraints are introduced to transfer knowledge between the auxiliary network and the main network. Then， a soft-label guidance strategy is proposed to further improve the accuracy of ship detection. The experimental results show that compared with the existing mainstream methods， the proposed method demonstrates better performance on DOTA and xView datasets， overcoming the limitation of a single model and providing a new solution idea for target detection in remote sensing images.

Key words: remote sensing, ship detection, multi-scale representation, multiple models collaboration, knowledge fusion

CLC Number:

Xinlin XIAO, Weichao SHI, Xiangtao ZHENG, Yueming GAO, Xiaoqiang LU. Multiple models collaboration for ship detection[J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(14): 630241-630241.

Figures/Tables 9

Fig.1

Fig.2

Fig.3

Table 1

Table 2

Table 3

Table 4

Table 5

Fig.4

References 40

1	REN S Q， HE K M， GIRSHICK R， et al. Faster R-CNN： Towards real-time object detection with region proposal networks［C］∥ Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1. New York： ACM， 2015： 91–99.
2	ZHANG Y Y， LU T W. RecFRCN： Few-shot object detection with recalibrated faster R-CNN［J］. IEEE Access， 2023， 11： 121109-121117.
3	FAN J Y， LEE J， JUNG I， et al. Improvement of object detection based on faster R-CNN and YOLO［C］∥2021 36th International Technical Conference on Circuits/Systems， Computers and Communications （ITC-CSCC）. Piscataway： IEEE Press， 2021： 1-4.
4	KÖREZ A， BARISCI N. Object detection with low capacity GPU systems using improved faster R-CNN［J］. Applied Sciences， 2019， 10（1）： 83.
5	周兵，李润鑫，尚振宏，等. 基于改进的Faster R-CNN目标检测算法［J］. 激光与光电子学进展， 2020， 57（10）： 101009.
	ZHOU B， LI R X， SHANG Z H， et al. Object detection algorithm based on improved faster R-CNN［J］. Laser & Optoelectronics Progress， 2020， 57（10）： 101009 （in Chinese）.
6	ABBAS S M， SINGH D S N. Region-based object detection and classification using faster R-CNN［C］∥ 2018 4th International Conference on Computational Intelligence & Communication Technology （CICT）. Piscataway： IEEE Press， 2018： 1-6.
7	GAVRILESCU R， ZET C， FOȘALĂU C， et al. Faster R-CNN： An approach to real-time object detection［C］∥ 2018 International Conference and Exposition on Electrical and Power Engineering （EPE）. Piscataway： IEEE Press， 2018： 165-168.
8	REN Y， ZHU C R， XIAO S P. Small object detection in optical remote sensing images via modified faster R-CNN［J］. Applied Sciences， 2018， 8（5）： 813.
9	RAKHI A M， DHORAJIYA A P， SARANYA P. Enhanced mask-RCNN for ship detection and segmentation［C］∥International Conference on Ubiquitous Computing and Intelligent Information Systems. Singapore： Springer， 2022： 199-210.
10	FANG Y X， DAI Y， HE G L， et al. A mask RCNN based automatic reading method for pointer meter［C］∥2019 Chinese Control Conference （CCC）. Piscataway： IEEE Press， 2019： 8466-8471.
11	ZHANG F， WANG X Y， ZHOU S L， et al. Arbitrary-oriented ship detection through center-head point extraction［J］. IEEE Transactions on Geoscience and Remote Sensing， 2022， 60： 5612414.
12	HUANG B， XU T F， LUO Y X， et al. A novel nearest feature learning classifier for ship target detection in optical remote sensing images［C］∥International Conference in Communications， Signal Processing， and Systems. Singapore： Springer， 2019： 600-606.
13	NAJIBI M， SINGH B， DAVIS L. AutoFocus： Efficient multi-scale inference［C］∥2019 IEEE/CVF International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2019： 9744-9754.
14	QUAN Y， ZHANG D， ZHANG L Y， et al. Centralized feature pyramid for object detection［J］. IEEE Transactions on Image Processing， 2023， 32： 4341-4354.
15	CHANG Y J， WU Y D， LIAO Y L， et al. Modified YOLO network model for metaphase cell detection in antinuclear antibody images［J］. Engineering Applications of Artificial Intelligence， 2024， 127： 107392.
16	XING Y G， YI Z P， LIANG Z X， et al. Edge-on low-surface-brightness galaxy candidates detected from SDSS images using YOLO［J］. The Astrophysical Journal Supplement Series， 2023， 269（2）： 59.
17	CHEN L Q， SHI W X， FAN C E， et al. A novel coarse-to-fine method of ship detection in optical remote sensing images based on a deep residual dense network［J］. Remote Sensing， 2020， 12（19）： 3115.
18	SONG W S， YAN D M， YAN J， et al. Ship detection and identification in SDGSAT-1 glimmer images based on the glimmer YOLO model［J］. International Journal of Digital Earth， 2023， 16（2）： 4687-4706.
19	WANG J， PAN Q R， LU D H， et al. An efficient ship-detection algorithm based on the improved YOLOv5［J］. Electronics， 2023， 12（17）： 3600.
20	HA H Y， FLEITES F， CHEN S C. Building multi-model collaboration in detecting multimedia semantic concepts［C］∥Proceedings of the 9th IEEE International Conference on Collaborative Computing： Networking， Applications and Worksharing. Piscataway： IEEE Press， 2013： 205-212.
21	ZHANG Y N， TONG X M， YANG T， et al. Multi-model estimation based moving object detection for aerial video［J］. Sensors， 2015， 15（4）： 8214-8231.
22	MA W P， GUO Q Q， WU Y， et al. A novel multi-model decision fusion network for object detection in remote sensing images［J］. Remote Sensing， 2019， 11（7）： 737.
23	XU J， WANG W， WANG H Y， et al. Multi-model ensemble with rich spatial information for object detection［J］. Pattern Recognition， 2020， 99： 107098.
24	KUNDID VASIĆ M， PAPIĆ V. Multimodel deep learning for person detection in aerial images［J］. Electronics， 2020， 9（9）： 1459.
25	YANG D F， SOLIHIN M I， ZHAO Y W， et al. A review of intelligent ship marine object detection based on RGB camera［J］. IET Image Processing， 2024， 18（2）： 281-297.
26	FINGAS M F， BROWN C E. Review of ship detection from airborne platforms［J］. Canadian Journal of Remote Sensing， 2001， 27（4）： 379-385.
27	YASIR M， WAN J H， XU M M， et al. Ship detection based on deep learning using SAR imagery： A systematic literature review［J］. Soft Computing， 2023， 27（1）： 63-84.
28	TEIXEIRA E， ARAUJO B， COSTA V， et al. Literature review on ship localization， classification， and detection methods based on optical sensors and neural networks［J］. Sensors， 2022， 22（18）： 6879.
29	LIN Y， WANG C F， CHANG C Y， et al. An efficient framework for counting pedestrians crossing a line using low-cost devices： The benefits of distilling the knowledge in a neural network［J］. Multimedia Tools and Applications， 2021， 80（3）： 4037-4051.
30	XIA G S， BAI X， DING J， et al. DOTA： A large-scale dataset for object detection in aerial images［C］∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE Press， 2018： 3974-3983.
31	BYNUM L， DOSTER T， EMERSON T H， et al. Rotational equivariance for object classification using xView［C］∥IGARSS 2020 - 2020 IEEE International Geoscience and Remote Sensing Symposium. Piscataway： IEEE Press， 2020： 3684-3687.
32	LI K， WAN G， CHENG G， et al. Object detection in optical remote sensing images： A survey and a new benchmark［J］. ISPRS Journal of Photogrammetry and Remote Sensing， 2020， 159： 296-307.
33	ZHANG Y L， YUAN Y， FENG Y C， et al. Hierarchical and robust convolutional neural network for very high-resolution remote sensing object detection［J］. IEEE Transactions on Geoscience and Remote Sensing， 2019， 57（8）： 5535-5548.
34	ZOU Z X， SHI Z W. Random access memories： A new paradigm for target detection in high resolution aerial remote sensing images［J］. IEEE Transactions on Image Processing， 2018， 27（3）： 1100-1111.
35	NEUBECK A， VAN GOOL L. Efficient non-maximum suppression［C］∥18th International Conference on Pattern Recognition （ICPR’06）. Piscataway： IEEE Press， 2006： 850-855.
36	BODLA N， SINGH B， CHELLAPPA R， et al. Soft-NMS—improving object detection with one line of code［C］∥2017 IEEE International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2017： 5562-5570.
37	SOLOVYEV R， WANG W M， GABRUSEVA T. Weighted boxes fusion： Ensembling boxes from different object detection models［J］. Image and Vision Computing， 2021， 107： 104117.
38	WANG C Y， BOCHKOVSKIY A， LIAO H Y M. YOLOv7： Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors［C］∥2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2023： 7464-7475.
39	GE Z， LIU S T， WANG F， et al. YOLOX： Exceeding YOLO series in 2021［DB/OL］. arXiv preprint： 2107.08430， 2021.
40	CHEN Z， LIU C， FILARETOV V， et al. Multi-scale ship detection algorithm based on YOLOv7 for complex scene SAR images［J］. Remote Sensing， 2023， 15（8）： 2071.

网络	三元关系约束	知识引导	DOTA		xView
网络	三元关系约束	知识引导	AP_0.5	AP_0.95	AP_0.5	AP_0.95
主网络			0.559	0.337	0.440	0.197
	√		0.657	0.434	0.531	0.282
		√	0.612	0.406	0.526	0.277
	√	√	0.735	0.493	0.551	0.299

方法	DOTA		xView
方法	AP_0.5	AP_0.95	AP_0.5	AP_0.95
联合引导	0.712	0.478	0.546	0.291
融合引导	0.735	0.493	0.551	0.299

方法	DOTA		xView
方法	AP_0.5	AP_0.95	AP_0.5	AP_0.95
主+辅1	0.589	0.377	0.451	0.206
主+辅2	0.568	0.365	0.467	0.243
主+辅3	0.565	0.358	0.479	0.256
主+辅1+辅2	0.636	0.410	0.497	0.269
主+辅1+辅3	0.649	0.418	0.516	0.272
主+辅2+辅3	0.631	0.403	0.521	0.275
本文方法	0.735	0.493	0.551	0.299

方法		DOTA		xView
方法		AP_0.5	AP_0.95	AP_0.5	AP_0.95
仅主网络		0.559	0.337	0.440	0.197
数据集成	DOTA/xView with LEVIR+DIOR+HRRSD	0.708	0.462	0.528	0.281
模型集成	NMS ^［35］	0.642	0.426	0.537	0.285
	Soft-NMS ^［36］	0.647	0.431	0.540	0.287
	WBF ^［37］	0.673	0.442	0.545	0.290
模型交互	MMDF-Net ^［20］	0.631	0.401	0.469	0.241
	SSMD ^［21］	0.679	0.434	0.461	0.238
	本文方法	0.735	0.493	0.551	0.299

方法	DOTA		xView
方法	AP_0.5	AP_0.95	AP_0.5	AP_0.95
YOLOv7 ^［38］	0.519	0.315	0.416	0.183
YOLOX ^［39］	0.571	0.337	0.431	0.201
CSD-YOLO ^［40］	0.594	0.362	0.457	0.224
AutoFocus ^［13］	0.543	0.321	0.403	0.177
CFP ^［14］	0.511	0.304	0.348	0.133
本文方法	0.735	0.493	0.551	0.299

Multiple models collaboration for ship detection

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 40

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	Qichang ZHAO, Yiquan WU, Yubin YUAN. Progress of ship detection and recognition methods in optical remote sensing images [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(8): 29025-029025.
[2]	Ziling WANG, Zhenyu XIONG, Lucheng YANG, Ruining YANG, Linzhou HUANG. Spaceborne SAR ship target recognition network guided by AIS and optical remote sensing images [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(2): 328672-328672.
[3]	Yanfang LIU, Jiayu SHE, Qiufan YUAN, Rui ZHOU, Naiming QI. Real⁃time small target detection networks for UAV remote sensing [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(14): 630119-630119.
[4]	Liqun CHEN, Xu ZOU, Lei ZHANG, Yingpan ZHU, Gang WANG, Jinyong CHEN. On⁃board intelligent target detection technology based on domestic commercial components [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(S2): 728860-728860.
[5]	Zhenyu XIONG, Yaqi CUI, Kai DONG, Mengyang LI, Wei XIONG. Interpretable fusion association network for multi-source remote sensing ship target based on attribute guidance [J]. Acta Aeronautica et Astronautica Sinica, 2023, 44(22): 627476-627476.
[6]	WANG Ziling, XIONG Zhenyu, GU Xiangqi. Correlation learning algorithm of visible light and SAR cross modal remote sensing images [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(S1): 727239-727239.
[7]	SU Hang, XU Congan, YAO Libo, LI Jianwei, LING Qing, GAO Long. A lightweight oriented ship detection method in SAR images [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022, 43(S1): 726922-726922.
[8]	LIU Jingyu, CUI Yufu, XU Meng, HE Hongyan, YIN Huan, WANG Yu. Image restoration method for TDICCD camera with micro vibration [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2018, 39(S1): 722230-722230.
[9]	YANG Junli, JIANG Zhiguo, ZHOU Quan, ZHANG Haopeng, SHI Jun. Remote sensing image semantic labeling based on conditional random field [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2015, 36(9): 3069-3081.
[10]	YE Dong, TU Yuanyuan, SUN Zhaowei. Sliding mode control for nonparallel-ground-track imaging using Chebyshev neural network [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2015, 36(9): 3092-3104.
[11]	MENG Guang, ZHOU Xubin. Progress review of satellite micro-vibration and control [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2015, 36(8): 2609-2619.
[12]	ZHAO Qinglei, LONG Kehui, HAN Chengshan, LI Xiangzhi, HUANG Liang. Space TDICCD Camera Imaging Parameters In-orbit Re-injection [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2014, 35(8): 2271-2278.
[13]	MENG Gang, HE Jie, BAO Li, WANG Jiantao, YAN Sunzhen, XU Jinping. Airport Runway Detection Based on Line Feature Detection of Partitioned Remote Sensing Images [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2014, 35(7): 1957-1965.
[14]	LI Jun, LI Jun, ZHONG Zhinong, JING Ning, HU Weidong. Space-air Resources Multi-phase Cooperation Task Planning Approach Based on Heterogeneous MAS Model [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2013, 34(7): 1682-1697.
[15]	LIU Zhanchao, FANG Jiancheng. Online Calibration of POS Error Based on Double Strapdown Algorithm [J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2012, 33(9): 1679-1687.