动态亮度重建的无人机可见光-红外融合目标检测

doi:10.7527/S1000-6893.2025.31968

Abstract

Abstract:

The visible-infrared fusion object detection of Unmanned Aerial Vehicles （UAV） has important application value in military and civilian fields such as disaster rescue， security monitoring， and battlefield reconnaissance. However， under low illumination conditions， existing fusion strategies have several limitations， including ignoring uneven lighting in different areas of the same scene and over -reliance on infrared modalities， which results in the potential rich semantic information of visible images in low illumination conditions. In addition， low light further exacerbates the difficulty of cross modal fusion. To address the above problems， a dynamic brightness reconstruction for UAV visible-infrared fusion object detection method is proposed. Firstly， a Super pixel Dynamic Illumination aware Mask （SDIM） module was designed using prior local illumination information. By simulating the dependence of real scenes on different modalities and introducing superpixel information， the problem of object edge feature loss in existing methods was solved. Secondly， considering the problem of feature degradation in low light visible images， a Low Illumination Image Enhancement （LIIE） module was designed to achieve end-to-end optimization of visible image key semantic adaptive enhancement for detection tasks. Finally， a Multi-Scale Feature Cross-Attention Fusion （MFCF） module was designed to address the fusion conflicts caused by cross modal feature heterogeneity. The module constructs a bimodal feature interaction space through a hierarchical cross attention mechanism and adaptively fuses multi-scale features using a dynamic weight allocation strategy. Based on the typical visible-infrared fusion object detection datasets DroneVehicle and VEDAI， the experimental results verified the effectiveness and robustness of the proposed method in visible-infrared fusion target detection tasks under low illumination conditions. Specifically， compared with existing advanced fusion detection algorithms， the proposed method improved the average accuracy （mAP） by 2.3% and 2.2% respectively while maintaining a low number of parameters， and compared with the widely used single-mode YOLOv8 algorithm， the mAP has increased by up to 12.9%. In addition， cross scene experimental results based on the LIS real low illumination dataset further validated the excellent generalization capability of the proposed method.

Key words: low illumination, UAV, visible-infrared, deep learning, object detection

CLC Number:

V279

Kui LIU, Hao SUN, Han WU, Kefeng JI, Gangyao KUANG. Dynamic brightness reconstruction for UAV visible-infrared fusion object detection[J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(23): 631968.

Figures/Tables 14

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Table 1

Table 2

Table 3

Fig.6

Fig.7

Table 4

Table 5

Table 6

Fig.8

References 51

[1]	SUN Y M， CAO B， ZHU P F， et al. Drone-based RGB-infrared cross-modality vehicle detection via uncertainty-aware learning［J］. IEEE Transactions on Circuits and Systems for Video Technology， 2022， 32（10）： 6700-6713.
[2]	ZHU Y H， SUN X Y， WANG M， et al. Multi-modal feature pyramid transformer for RGB-infrared object detection［J］. IEEE Transactions on Intelligent Transportation Systems， 2023， 24（9）： 9984-9995.
[3]	吴一全，童康. 基于深度学习的无人机航拍图像小目标检测研究进展［J］. 航空学报， 2025， 46（3）： 181-207.
	WU Y Q， TONG K. Research advances on deep learning-based small object detection in UAV aerial images［J］. Acta Aeronautica et Astronautica Sinica， 2025， 46（3）： 181-207 （in Chinese）.
[4]	SHANG X P， LI N N， LI D J， et al. CCLDet： A cross-modality and cross-domain low-light detector［J］. IEEE Transactions on Intelligent Transportation Systems， 2025， 26（3）： 3284-3294.
[5]	LI C Y， SONG D， TONG R F， et al. Illumination-aware faster R-CNN for robust multispectral pedestrian detection［J］. Pattern Recognition， 2019， 85： 161-171.
[6]	LIU X Y， QI J H， CHEN C， et al. Relation-aware weight sharing in decoupling feature learning network for UAV RGB-infrared vehicle re-identification［J］. IEEE Transactions on Multimedia， 2024， 26： 9839-9853.
[7]	WANG J P， XU C A， ZHAO C H， et al. Multimodal object detection of UAV remote sensing based on joint representation optimization and specific information enhancement［J］. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing， 2024， 17： 12364-12373.
[8]	YUAN M X， SHI X R， WANG N， et al. Improving RGB-infrared object detection with cascade alignment-guided transformer［J］. Information Fusion， 2024， 105： 102246.
[9]	GUAN D Y， CAO Y P， YANG J X， et al. Fusion of multispectral data through illumination-aware deep neural networks for pedestrian detection［J］. Information Fusion， 2019， 50： 148-157.
[10]	WANG Q W， CHI Y K， SHEN T， et al. Improving RGB-infrared object detection by reducing cross-modality redundancy［J］. Remote Sensing， 2022， 14（9）： 2020.
[11]	SUN X， YU Y H， CHENG Q. Low-rank multimodal remote sensing object detection with frequency filtering experts［J］. IEEE Transactions on Geoscience and Remote Sensing， 2024， 62： 5637114.
[12]	ZHAO D W， SHAO F M， ZHANG S， et al. Advanced object detection in low-light conditions： Enhancements to YOLOv7 framework［J］. Remote Sensing， 2024， 16（23）： 4493.
[13]	HUI Y M， WANG J， LI B. WSA-YOLO： Weak-supervised and adaptive object detection in the low-light environment for YOLOV7［J］. IEEE Transactions on Instrumentation and Measurement， 2024， 73： 2507012.
[14]	MORAWSKI I， CHEN Y， LIN Y S， et al. GenISP： Neural ISP for low-light machine cognition［C］∥2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops （CVPRW）. Piscataway： IEEE Press， 2022.
[15]	ELGUEBALY T， BOUGUILA N. Finite asymmetric generalized Gaussian mixture models learning for infrared object detection［J］. Computer Vision and Image Understanding， 2013， 117（12）： 1659-1671.
[16]	江泽涛，翟丰硕，钱艺，等. 结合特征增强和多尺度感受野的低照度目标检测［J］. 计算机研究与发展， 2023， 60（4）： 903-915.
	JIANG Z T， ZHAI F S， QIAN Y， et al. Low illumination object detection combined with feature enhancement and MultiScale receptive field［J］. Journal of Computer Research and Development， 2023， 60（4）： 903-915 （in Chinese）.
[17]	JIA X Y， ZHU C， LI M Z， et al. LLVIP： A visible-infrared paired dataset for low-light vision［C］∥2021 IEEE/CVF International Conference on Computer Vision Workshops （ICCVW）. Piscataway： IEEE Press， 2021： 3489-3497.
[18]	LOH Y P， CHAN C S. Getting to know low-light images with the Exclusively Dark dataset［J］. Computer Vision and Image Understanding， 2019， 178： 30-42.
[19]	PENG D X， DING W， ZHEN T. A novel low light object detection method based on the YOLOv5 fusion feature enhancement［J］. Scientific Reports， 2024， 14： 4486.
[20]	LIU S W， HE H， ZHANG Z C， et al. LI-YOLO： An object detection algorithm for UAV aerial images in low-illumination scenes［J］. Drones， 2024， 8（11）： 653.
[21]	江泽涛，肖芸，张少钦，等. 基于Dark-YOLO的低照度目标检测方法［J］. 计算机辅助设计与图形学学报， 2023， 35（3）： 441-451.
	JIANG Z T， XIAO Y， ZHANG S Q， et al. Low-illumination object detection method based on dark-YOLO［J］. Journal of Computer-Aided Design & Computer Graphics， 2023， 35（3）： 441-451 （in Chinese）.
[22]	DU Z P， SHIT M， DENG J K. Boosting object detection with zero-shot day-night domain adaptation［C］∥2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2024： 12666-12676.
[23]	WANG W J， WANG X H， YANG W H， et al. Unsupervised face detection in the dark［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2023， 45（1）： 1250-1266.
[24]	SASAGAWA Y， NAGAHARA H. YOLO in the dark-domain adaptation method for merging multiple models［C］∥Computer Vision-ECCV 2020. Cham： Springer International Publishing， 2020： 345-359.
[25]	GUO C L， LI C Y， GUO J C， et al. Zero-reference deep curve estimation for low-light image enhancement［C］∥2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2020.
[26]	ZHANG L， LIU Z Y， ZHU X Y， et al. Weakly aligned feature fusion for multimodal object detection［J］. IEEE Transactions on Neural Networks and Learning Systems， 2025， 36（3）： 4145-4159.
[27]	LIU Y， JIANG W S. Frequency mining and complementary fusion network for RGB-infrared object detection［J］. IEEE Geoscience and Remote Sensing Letters， 2024， 21： 5004605.
[28]	RAZAKARIVONY S， JURIE F. Vehicle detection in aerial imagery： A small target detection benchmark［J］. Journal of Visual Communication and Image Representation， 2016， 34： 187-203.
[29]	ZHANG J Q， LEI J， XIE W Y， et al. SuperYOLO： Super resolution assisted object detection in multimodal remote sensing imagery［J］. IEEE Transactions on Geoscience and Remote Sensing， 2023， 61： 5605415.
[30]	JIN S P， YU B B， JING M H， et al. DarkVisionNet： Low-light imaging via RGB-NIR fusion with deep inconsistency prior［J］. Proceedings of the AAAI Conference on Artificial Intelligence， 2022， 36（1）： 1104-1112.
[31]	SHARMA M， DHANARAJ M， KARNAM S， et al. YOLOrs： Object detection in multimodal remote sensing imagery［J］. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing， 2020， 14： 1497-1508.
[32]	CHEN J， DING J F， MA J Y. HitFusion： Infrared and visible image fusion for high-level vision tasks using transformer［J］. IEEE Transactions on Multimedia， 2024， 26： 10145-10159.
[33]	李峻宇，刘乾坤，付莹. 融合注意力机制的红外小目标检测［J］. 航空学报， 2024， 45（14）： 628959.
	LI J Y， LIU Q K， FU Y. Infrared small object detection based on attention mechanism［J］. Acta Aeronautica et Astronautica Sinica， 2024， 45（14）： 628959 （in Chinese）.
[34]	ACHANTA R， SHAJI A， SMITH K， et al. SLIC superpixels compared to state-of-the-art superpixel methods［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2012， 34（11）： 2274-2282.
[35]	HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition［C］∥2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2016.
[36]	CHEN L W， GU L， LI L， et al. Frequency dynamic convolution for dense image prediction［C］∥2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2025.
[37]	CHEN L W， FU Y， WEI K X， et al. Instance segmentation in the dark［J］. International Journal of Computer Vision， 2023， 131（8）： 2198-2218.
[38]	HAN J M， DING J， LI J， et al. Align deep features for oriented object detection［J］. IEEE Transactions on Geoscience and Remote Sensing， 2021， 60： 5602511.
[39]	YANG Z， LIU S H， HU H， et al. RepPoints： Point set representation for object detection［C］∥2019 IEEE/CVF International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2019： 9656-9665.
[40]	YANG X， YAN J C， FENG Z M， et al. R3Det： Refined single-stage detector with feature refinement for rotating object［J］. Proceedings of the AAAI Conference on Artificial Intelligence， 2021， 35（4）： 3163-3171.
[41]	REN S Q， HE K M， GIRSHICK R， et al. Faster R-CNN： Towards real-time object detection with region proposal networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（6）： 1137-1149.
[42]	XIE X X， CHENG G， WANG J B， et al. Oriented R-CNN for object detection［C］∥2021 IEEE/CVF International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2021： 3500-3509.
[43]	JOCHER G， QIU J， CHAURASIA A. Ultralytics YOLO［EB/OL］. （2023-01）［2025-02-28］. .
[44]	ZHANG H， LI F， LIU S L， et al. DINO： DETR with improved denoising anchor boxes for end-to-end object detection［DB/OL］. arXiv：， 2022.
[45]	TIAN Z， SHEN C H， CHEN H， et al. FCOS： Fully convolutional one-stage object detection［C］∥2019 IEEE/CVF International Conference on Computer Vision （ICCV）. Piscataway： IEEE Press， 2019： 9626-9635.
[46]	CHEN L W， FU Y， GU L， et al. Frequency-aware feature fusion for dense image prediction［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2024， 46（12）： 10763-10780.
[47]	CHEN L W， GU L， ZHENG D Z， et al. Frequency-adaptive dilated convolution for semantic segmentation［C］∥2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. Piscataway： IEEE Press， 2024： 3414-3425.
[48]	SHEN J F， CHEN Y F， LIU Y， et al. ICAFusion： Iterative cross-attention guided feature fusion for multispectral object detection［J］. Pattern Recognition， 2024， 145： 109913.
[49]	ZHOU K L， CHEN L S， CAO X. Improving multispectral pedestrian detection by addressing modality imbalance problems［C］∥Computer Vision-ECCV 2020. Cham： Springer， 2020： 787-803.
[50]	ZHANG J Q， CAO M X， XIE W Y， et al. E2E-MFD： Towards end-to-end synchronous multimodal fusion detection［C］∥NIPS’24： Proceedings of the 38th International Conference on Neural Information Processing Systems. Curran Associates Inc.， 2024： 52296-52322.
[51]	YUAN M X， WEI X X. C²Former： Calibrated and complementary transformer for RGB-infrared object detection［J］. IEEE Transactions on Geoscience and Remote Sensing， 2024， 62： 5403712.

方法	汽车	面包车	公交车	货车	卡车	mAP50	Params.（M）	GFLOPs	模态
S²A-Net	0.866	0.425	0.757	0.260	0.207	0.503	36.2	197.6	RGB
RepPoints	0.876	0.390	0.622	0.244	0.222	0.471	36.6	194.3
R³Det	0.801	0.318	0.718	0.220	0.174	0.446	41.7	330.7
Faster R-CNN	0.892	0.647	0.874	0.475	0.438	0.665	41.1	198.4
Oriented R-CNN	0.893	0.673	0.890	0.493	0.437	0.677	41.1	198.5
YOLOv8	0.967	0.720	0.943	0.557	0.535	0.744	3.1	8.4
S²A-Net	0.896	0.365	0.779	0.155	0.242	0.488	36.2	197.6	IR
RepPoints	0.894	0.406	0.609	0.270	0.285	0.493	36.6	194.3
R³Det	0.892	0.346	0.776	0.231	0.297	0.508	41.7	330.7
Faster R-CNN	0.902	0.638	0.884	0.446	0.488	0.672	41.1	198.4
Oriented R-CNN	0.903	0.704	0.892	0.464	0.546	0.702	41.1	198.5
YOLOv8	0.983	0.769	0.949	0.565	0.644	0.782	3.1	8.4
UA-CMDet	0.875	0.871	0.607	0.380	0.468	0.640	138.7		RGB+IR
ICAFusion	0.816	0.857	0.560	0.318	0.333	0.577	120.2	180.0
MBNet	0.901	0.536	0.888	0.624	0.644	0.719
LF-MDet	0.822	0.866	0.736	0.570	0.596	0.718	38.7	77.7
E2E-MFD	0.903	0.898	0.646	0.631	0.793	0.774	31.3	105.5
本文算法	0.985	0.583	0.962	0.650	0.804	0.797	30.2	143.6

方法	汽车	卡车	船	拖拉机	露营车	面包车	皮卡车	其他	mAP50	Params.（M）	GFLOPs	模态
R3Det	0.658	0.233	0.251	0.150	0.454	0.343	0.524	0.202	0.352	41.7	330.7	RGB
Faster R-CNN	0.869	0.705	0.710	0.668	0.832	0.738	0.823	0.537	0.735	41.1	198.4
DINO	0.556	0.437	0.409	0.483	0.577	0.594	0.530	0.281	0.738	47.6	284.0
YOLOv8	0.846	0.715	0.532	0.675	0.746	0.630	0.753	0.496	0.675	3.1	8.4
R3Det	0.670	0.182	0.270	0.252	0.467	0.219	0.599	0.345	0.375	41.7	330.7	IR
Faster R-CNN	0.859	0.819	0.784	0.500	0.789	0.746	0.836	0.364	0.712	41.1	198.4
DINO	0.509	0.427	0.384	0.399	0.563	0.518	0.551	0.283	0.688	47.6	284.0
YOLOv8	0.857	0.769	0.419	0.519	0.770	0.740	0.762	0.367	0.650	3.1	8.4
SuperYOLO	0.911	0.702	0.602	0.804	0793	0.765	0.857	0.573	0.751	4.8	18.0	RGB+IR
ICAFusion	0.837	0.486	0.459	0.713	0.699	0.670	0.776	0.457	0.637	120.2	311.5
C2Former	0.872	0.774	0.714	0.729	0.827	0.752	0.807	0.584	0.757	101.0	430.5
本文算法	0.919	0.821	0.713	0.798	0.764	0.849	0.845	0.523	0.779	30.2	143.6

方法	自行车	汽车	电动车	公交车	水瓶	椅子	显示器	餐桌	mAP50	Params.（M）	GFLOPs	模态
FCOS	0.431	0.822	0.421	0.567	0.477	0.389	0.179	0.330	0.450	32.1	191.5	RGB
FreqFusion	0.586	0.528	0.527	0.571	0.757	0.864	0.394	0.581	0.601	46.9	103.3
FADC	0.588	0.520	0.495	0.536	0.725	0.866	0.357	0.563	0.581	41.4	201.7
Faster R-CNN	0.547	0.861	0.552	0.703	0.498	0.507	0.325	0.500	0.561	41.2	201.7
YOLOv8	0.619	0.849	0.612	0.810	0.550	0.505	0.413	0.650	0.626	3.1	8.4
w/ LIIE	0.637	0.864	0.606	0.814	0.575	0.515	0.410	0.631	0.631	5.7	92.0

SDIM	LIIE	MFCF	mAP50
√			0.787
	√		0.788
		√	0.774
√	√		0.792
√	√	√	0.797

模块	迭代次数	mAP50
LIIE	2	0.614
	4	0.627
	8	0.631
	10	0.622
	16	0.612

Dynamic brightness reconstruction for UAV visible-infrared fusion object detection

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 14

References 51

Related Articles 15

Recommended Articles

Metrics

Comments

编码层融合	解码层融合	编码-解码融合	mAP50
相加	相加	相加	0.792
相加	相加	交叉注意力	0.789
交叉注意力	交叉注意力	相加	0.797
交叉注意力	交叉注意力	交叉注意力	0.784

[1]	Jiong HE, Binwu REN, Siliang DU, Yousong XU, Bo WANG. Adaptive attitude control for tilt-quadrotor UAV based on ADRC-RBF [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(S1): 732189-732189.
[2]	Xiangyu YU, Wen LI, Jie YAN, Shizhe LIANG. Simulation research on thermal management system of fuel cell for liquid hydrogen powered UAV [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(9): 630964-630964.
[3]	Pengqian YANG, Yutong CHEN, Junhui LIU, Jiehao YANG, Jiayuan SHAN, Shijun SUN. Aerodynamic and operational characteristics analysis for tandem wing cargo UAV at high angle of attack [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(9): 131056-131056.
[4]	Rongzu LI, Li LIU, Dun YANG. Optimal design of hydrogen-powered UAV based on multi-source domain fusion surrogate model [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(9): 630979-630979.
[5]	Jianyu XU, Li ZHOU, Zhanxue WANG, Jie SHI, Hao SHI. Calculation method for hypersonic plume infrared radiation based on a fast line-by-line calculation model [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(8): 630778-630778.
[6]	Kaifang WAN, Zhilin WU, Yunhui WU, Haozhi QIANG, Yibo WU, Bo LI. Cooperative location of multiple UAVs with deep reinforcement learning in GPS-denied environment [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(8): 331024-331024.
[7]	Lingfeng JIANG, Xinkai LI, Hai ZHANG, Hanwei LI, Hongli ZHANG. Mapless navigation of UAVs in dynamic environments based on an improved TD3 algorithm [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(8): 331035-331035.
[8]	Lingjie MENG, Hongguang LI, Xinjun LI. SAR image simulation method guided by geomorphic category information [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(7): 331003-331003.
[9]	Jinwu XIANG, Kai MA, Zi KAN, Daochun LI, Kexin ZHENG, Hanxuan CHEN. Review of key technologies for hydrogen powered unmanned aerial vehicles [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(5): 531603-531603.
[10]	Qishuai DING, Bangjun LEI, Zhengping WU. A lightweight single object tracking algorithm for UAVs based on Siamese network [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(4): 330925-330925.
[11]	Zhihao ZHAO, Zhaohua YANG, Yun WU, Yuanjin YU. Single-photon counting imaging denoising method based on deep learning in low-light environment [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(3): 630531-630531.
[12]	Nuo MA, Shechun WEI, Junhui MENG, Qingyang LIU, Yusheng LEI. Flow field characteristics and dynamics of internal supply chamber separating from UAV considering effect of deceleration parachutes [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(3): 130755-130755.
[13]	Yiquan WU, Kang TONG. Research advances on deep learning-based small object detection in UAV aerial images [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(3): 30848-030848.
[14]	Wei HUANG, Jiahao PAN, Chu HE. Wavelet time-frequency localization-based model compression for UAV object detection [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(23): 631952-631952.
[15]	Tianqi FAN, Zhengxia ZOU, Zhenwei SHI. Typical remote sensing target detection with data synthesis based on reinforcement learning [J]. Acta Aeronautica et Astronautica Sinica, 2025, 46(23): 631955-631955.