融合注意力机制的红外小目标检测

Abstract

Abstract:

As the detection accuracy of salient objects in the image has been improved， the focus of research has gradually shifted to how to improve the accuracy of small object detection. However， existing object detection methods mainly study the general object detection using visible images as the input， and most small object detection methods are designed for visible images， leaving the small object detection in infrared images underexplored. Compared with standard scale objects， infrared small objects lack color information， which makes them more dependent on contextual information. In this paper， an infrared small object detection model is proposed based on the standard YOLOv5 model. The local information around small objects is effectively combined with global information by a Dynamic Contextual Information Extraction Module， which adapts to the subtle morphological changes of infrared small objects dynamically. A Channel-Detail Attention Module is designed to aggregate the channel and detail information of the infrared small objects to improve the accuracy of the regression. Considering the problem of loss of detailed features in the process of network convolution， features with new scales are upsampled and fused with shallow features to capture more detailed information of infrared small objects and avoid feature blending. To demonstrate the effectiveness of the proposed method， experiments are conducted on the public infrared datasets， including ITTD， IRSTD-1k， and NUAA-SIRST. The experimental results show that the proposed method outperforms the compared methods in terms of mAP by 5.1% on the ITTD dataset， and the mAP is also improved by 3.7% compared to that of the baseline method （i.e.， YOLOv5）. Results on the IRSTD-1k and NUAA-SIRST datasets also demonstrate the effectiveness of our design. An ablation study is performed to verify the effectiveness of different modules. The proposed infrared small object detection model is robust to the infrared small objects in complex backgrounds， which improves the accuracy and reduces the false alarm rate of infrared small object detection effectively.

Key words: deep learning, object detection, infrared small object detection, attention mechanism, YOLOv5

CLC Number:

Junyu LI, Qiankun LIU, Ying FU. Infrared small object detection based on attention mechanism[J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(14): 628959.

Figures/Tables 9

Fig.1

Fig.2

Fig.3

Fig.4

Table 1

Comparison of different models on ITTD dataset

网络模型	ITTD数据集		FPS/s	FLOPs/G	Params/M
网络模型	mAP	$F m$	FPS/s	FLOPs/G	Params/M
SSD^［4］	78.4	9.9	84.2	110.0	24.0
Faster R-CNN^［28］	81.8	10.0	33.1	91.0	41.1
Cascade R-CNN^［29］	80.8	12.9	22.6	91.1	68.9
Dynamic R-CNN^［30］	85.1	7.7	33.4	63.2	41.1
Sparse R-CNN^［31］	84.1	4.1	33.5	44.1	105.9
YOLOx^［32］	84.8	7.1	101.4	13.3	8.9
YOLOv5s	87.6	10.6	151.5	16.4	6.7
YOLOv7^［33］	85.9	17.9	51.2	104.8	35.4
IDSTD^［34］	80.6	5.0	48.1	65.6	36.1
DCFD^［35］	86.2	10.3	43.4	154.9	58.6
本文方法	91.3	6.6	93.4	19.2	7.2

Table 1

Table 2

Comparison of different models on IRSTD-1k and NUAA-SIRST datasets

网络模型	IRSTD-1k数据集		NUAA-SIRST数据集
网络模型	mAP	$F m$	mAP	$F m$
SSD^［4］	62.2	17.2	77.7	11.5
Faster R-CNN^［28］	79.4	16.1	78.6	6.2
Cascade R-CNN^［29］	80.0	16.3	87.0	10.4
Dynamic R-CNN^［30］	83.2	13.7	86.6	3.2
Sparse R-CNN^［31］	74.6	5.9	47.7	8.0
YOLOx^［32］	79.5	7.0	89.7	4.0
YOLOv5s	84.6	24.8	87.9	14.6
YOLOv7^［33］	80.9	28.5	86.2	17.7
IDSTD^［34］	84.5	26.6	81.0	23.6
DCFD^［35］	85.1	8.3	85.8	3.6
本文方法	87.2	25.9	91.1	10.6

Table 2

Fig.5

Table 3

Ablation experimental results

特征融合	通道-细节注意力模块	动态上下文信息提取模块	mAP	$F m$
×	×	×	87.6	10.6
√	×	×	88.9	7.2
√	√	×	90.4	6.9
√	×	√	90.1	7.1
√	√	√	91.3	6.6

Table 3

Table 4

References 36

1	BAI X Z， ZHOU F G. Analysis of new top-hat transformation and the application for infrared dim small target detection［J］. Pattern Recognition， 2010， 43（6）： 2145-2156.
2	PHILIP CHEN C L， LI H， WEI Y T， et al. A local contrast method for small infrared target detection［J］. IEEE Transactions on Geoscience and Remote Sensing， 2014， 52（1）： 574-581.
3	DAI Y M， WU Y Q. Reweighted infrared patch-tensor model with both nonlocal and local priors for single-frame small target detection［J］. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing， 2017， 10（8）： 3752-3767.
4	LIU W， ANGUELOV D， ERHAN D， et al. SSD： Single shot MultiBox detector［C］∥European Conference on Computer Vision. Cham： Springer， 2016： 21-37.
5	REDMON J， DIVVALA S， GIRSHICK R， et al. You only look once： Unified， real-time object detection［C］∥ 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2016： 779-788.
6	袁翔，程塨，李戈，等. 遥感影像小目标检测研究进展［J］. 中国图象图形学报， 2023， 28（6）： 1662-1684.
	YUAN X， CHENG G， LI G， et al. Progress in small object detection for remote sensing images［J］. Journal of Image and Graphics， 2023， 28（6）： 1662-1684 （in Chinese）.
7	ZHAO M X， CHENG L， YANG X， et al. TBC-Net： A real-time detector for infrared small target detection using semantic constraint［DB/OL］. arXiv preprint：2001.05852，2019.
8	CHENG G， YUAN X， YAO X W， et al. Towards large-scale small object detection： survey and benchmarks［DB/OL］. arXiv preprint： 2207.14096， 2022.
9	XU H， ZHONG S， ZHANG T X， et al. Multiscale multilevel residual feature fusion for real-time infrared small target detection［J］. IEEE Transactions on Geoscience and Remote Sensing， 2023， 61： 5002116.
10	WANG K W， DU S Y， LIU C X， et al. Interior attention-aware network for infrared small target detection［J］. IEEE Transactions on Geoscience and Remote Sensing， 2022， 60： 5002013.
11	BAI Y N， LI R M， GOU S P， et al. Cross-connected bidirectional pyramid network for infrared small-dim target detection［J］. IEEE Geoscience and Remote Sensing Letters， 2022， 19： 7506405.
12	CHENG G， YUAN X， YAO X W， et al. Towards large-scale small object detection： Survey and benchmarks［DB/OL］.arXiv preprint： 2207.14096， 2022.
13	李子豪，王正平，贺云涛. 基于自适应协同注意力机制的航拍密集小目标检测算法［J］. 航空学报， 2023， 44（13）： 327944.
	LI Z H， WANG Z P， HE Y T. Aerial-photography dense small target detection algorithm based on adaptive cooperative attention mechanism［J］. Acta Aeronautica et Astronautica Sinica， 2023， 44（13）： 327944 （in Chinese）.
14	冒国韬，邓天民，于楠晶. 基于多尺度分割注意力的无人机航拍图像目标检测算法［J］. 航空学报， 2023， 44（5）： 326738.
	MAO G T， DENG T M， YU N J. Object detection in UAV images based on multi-scale split attention［J］. Acta Aeronautica et Astronautica Sinica， 2023， 44（5）： 326738 （in Chinese）.
15	NOH J， BAE W， LEE W， et al. Better to follow， follow to be better： Towards precise supervision of feature super-resolution for small object detection［C］∥ 2019 IEEE/CVF International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2019： 9724-9733.
16	KISANTAL M， WOJNA Z， MURAWSKI J， et al. Augmentation for small object detection［DB/OL］. arXiv preprint：1902.07296， 2019.
17	LIN T Y， DOLLÁR P， GIRSHICK R， et al. Feature pyramid networks for object detection［C］∥ 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2017： 936-944.
18	CHEN C Y， LIU M Y， TUZEL O， et al. R-CNN for small object detection［C］∥LAI S H， LEPETIT V， NISHINO K， et al. Asian Conference on Computer Vision. Cham： Springer， 2017： 214-230.
19	JOCHER G， STOKEN A， BOROVEC J， et al. Ultralytics/yolov5： v5. 0-YOLO v5-P6 1280 models， AWS， Supervise.ly and YouTube integrations［EB/OL］. （2021-04-12）［2023-05-01］. .
20	DOSOVITSKIY A， BEYER L， KOLESNIKOV A， et al. An image is worth 16×16 words： Transformers for image recognition at scale［DB/OL］. arXiv preprint： 2010.11929，2020.
21	TIAN Y， FU Y， ZHANG J. Transformer-based under-sampled single-pixel imaging［J］. Chinese Journal of Electronics， 2023， 32（5）： 1151-1159.
22	LI M Y， FU Y， ZHANG Y L. Spatial-spectral transformer for hyperspectral image denoising［DB/OL］. arXiv preprint： 2211.14090， 2022.
23	DAI J F， QI H Z， XIONG Y W， et al. Deformable convolutional networks［C］∥ 2017 IEEE International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2017： 764-773.
24	WOO S， PARK J， LEE J Y， et al. CBAM： Convolutional block attention module［C］∥ European Conference on Computer Vision. Cham： Springer， 2018： 3-19.
25	傅瑞罡，范红旗，朱永锋，等. 面向空地应用的红外时敏目标检测跟踪数据集［J］. 中国科学数据（中英文网络版），2022， 7（2）： 203-218.
	FU R G， FAN H Q， ZHU Y F， et al. A dataset for infrared time-sensitive target detection and tracking for air-ground application［J］. China Scientific Data， 2022， 7（2）： 203-218 （in Chinese）.
26	ZHANG M J， ZHANG R， YANG Y X， et al. ISNet： Shape matters for infrared small target detection［C］∥ 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2022： 867-876.
27	DAI Y M， WU Y Q， ZHOU F， et al. Asymmetric contextual modulation for infrared small target detection［C］∥ 2021 IEEE Winter Conference on Applications of Computer Vision （WACV）. Piscataway： IEEE Press， 2021： 949-958.
28	REN S Q， HE K M， GIRSHICK R， et al. Faster R-CNN： Towards real-time object detection with region proposal networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（6）： 1137-1149.
29	CAI Z W， VASCONCELOS N. Cascade R-CNN： High quality object detection and instance segmentation［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2021， 43（5）： 1483-1498.
30	ZHANG H K， CHANG H， MA B P， et al. Dynamic R-CNN： Towards high quality object detection via dynamic training［C］∥VEDALDI A， BISCHOF H， BROX T， et al. European Conference on Computer Vision. Cham： Springer， 2020： 260-275.
31	SUN P Z， ZHANG R F， JIANG Y， et al. Sparse R-CNN： End-to-end object detection with learnable proposals［C］∥ 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2021： 14449-14458.
32	GE Z， LIU S T， WANG F， et al. YOLOX： Exceeding YOLO series in 2021［DB/OL］.arXiv preprint： 2107.08430， 2021.
33	WANG C Y， BOCHKOVSKIY A， LIAO H Y M. YOLOv7： Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors［C］∥ 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2023： 7464-7475.
34	DU J M， LU H Z， HU M F， et al. CNN-based infrared dim small target detection algorithm using target-oriented shallow-deep features and effective small anchor［J］. IET Image Processing， 2021， 15（1）： 1-15.
35	ZHANG Y， ZHANG Y， SHI Z G， et al. Design and training of deep CNN-based fast detector in infrared SUAV surveillance system［J］. IEEE Access， 2019， 7： 137365-137377.
36	HOU Q B， ZHOU D Q， FENG J S. Coordinate attention for efficient mobile network design［C］∥ 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2021： 13708-13717.

模块	mAP
CBAM模块^［24］	88.7
CA模块^［36］	89.4
通道-细节注意力模块	89.8

[1]	Jianyu XU, Li ZHOU, Zhanxue WANG, Jie SHI, Hao SHI. Calculation method for hypersonic plume infrared radiation based on a fast line-by-line calculation model [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(8): 630778-630778.
[2]	Lingjie MENG, Hongguang LI, Xinjun LI. SAR image simulation method guided by geomorphic category information [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(7): 331003-331003.
[3]	Zhihao ZHAO, Zhaohua YANG, Yun WU, Yuanjin YU. Single-photon counting imaging denoising method based on deep learning in low-light environment [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(3): 630531-630531.
[4]	Yiquan WU, Kang TONG. Research advances on deep learning-based small object detection in UAV aerial images [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(3): 30848-030848.
[5]	Yi ZHENG, Xianghong CHENG, Xingbang TANG, Yi CAO. Oriented detection algorithm for insulator and their defects from aerial images based on improved ReDet [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(18): 331825-331825.
[6]	Guixian QU, Dongyang LIU, Xu YANG, Tian QIU, Chuankai LIU, Shuiting DING, Shuzheng YUAN, Kan GUO. Remaining useful life prediction method based on temporal information enhancement of sensors [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(17): 231634-231634.
[7]	Xiaowei JIANG, Yiquan WU. Research progress of UAV aerial image mosaic methods [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(17): 331799-331799.
[8]	Lin CHEN, Xiwen GU, Zhiying CHEN, Zhuo ZHANG, Xiaoliang SUN. High-precision monocular vision pose measurement for large distance span in carrier landing guidance [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(15): 331568-331568.
[9]	Bin SUN, Hang YOU, Wenbo LI, Xiangrui LIU, Jiayi MA. Dual-band payload image fusion and its applications in low-altitude remote sensing [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(11): 531343-531343.
[10]	Fanteng MENG, Yong QIN, Jing CUI, Yunpeng WU, Zicheng ZHANG, Shaowei WEI. Unknown risk detection in external environment of railroad using UAV images [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(11): 531262-531262.
[11]	Shusheng CHEN, Muliang JIA, Jiahao LIN, Shiyi JIN, Zhenghong GAO, Yueqing WANG, Zhiqiang MA, Zheng LI, Chenlong DUAN, Jiawei LI. Empowering aircraft technology applications with generative models: Research progress and prospects [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(10): 631194-631194.
[12]	Weishi CHEN, Hongchuang NIU, Xin WANG, Jian WAN, Xianfeng LU, Jie ZHANG, Qingbin WANG. Review on multi-source detection technologies for birds and drones in airport clearance area [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(10): 31251-031251.
[13]	Jie LIN, Zhigong TANG, Weiqi QIAN, Yueqing WANG, Peng ZHANG, Weixia XU, Jie LIU. Research progress and prospects of aircraft aerodynamic design based on generative models [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(10): 631679-631679.
[14]	Yonghai WANG, Haoge LI, Jiaxin LI, Yi DUAN, Chuan TIAN, Lingxi GUO, Xusheng WU. Rapid aircraft shape generation based on deep learning [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(10): 631614-631614.
[15]	Jiaqi LIU, Rongqian CHEN, Jinhua LOU, Xu HAN, Hao WU, Yancheng YOU. Aerodynamic shape optimization of high-speed helicopter rotor airfoil based on deep learning [J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(9): 529828-529828.

Infrared small object detection based on attention mechanism

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 36

Related Articles 15

Recommended Articles

Metrics

Comments