航空学报 > 2025, Vol. 46 Issue (23): 631968-631968   doi: 10.7527/S1000-6893.2025.31968

干扰环境下无人机多源感知专栏

动态亮度重建的无人机可见光-红外融合目标检测

刘奎1,2, 孙浩1,2, 伍瀚1,2, 计科峰1,2(), 匡纲要1,2   

  1. 1.国防科技大学 电子科学学院,长沙 410073
    2.国防科技大学 电子信息系统复杂电磁环境效应国家重点实验室,长沙 410073
  • 收稿日期:2025-03-12 修回日期:2025-03-29 接受日期:2025-05-28 出版日期:2025-06-10 发布日期:2025-06-06
  • 通讯作者: 计科峰 E-mail:jikefeng@nudt.edu.cn
  • 基金资助:
    国家自然科学基金(61971426)

Dynamic brightness reconstruction for UAV visible-infrared fusion object detection

Kui LIU1,2, Hao SUN1,2, Han WU1,2, Kefeng JI1,2(), Gangyao KUANG1,2   

  1. 1.College of Electronic Science and Technology,National University of Defense Technology,Changsha 410073,China
    2.State Key Laboratory of Complex Electromagnetic Environment Effects on Electronics and Information System,National University of Defense Technology,Changsha 410073,China
  • Received:2025-03-12 Revised:2025-03-29 Accepted:2025-05-28 Online:2025-06-10 Published:2025-06-06
  • Contact: Kefeng JI E-mail:jikefeng@nudt.edu.cn
  • Supported by:
    National Natural Science Foundation of China(61971426)

摘要:

无人机可见光-红外融合目标检测在灾害救援、安防监控和战场侦察等军民用领域具有重要的应用价值。然而在低照度条件下,现有融合策略存在诸多不足,不仅忽略了同一场景内不同区域光照不均衡的问题,还过度依赖红外模态,导致低照度条件下的可见光图像潜在丰富的语义信息未被充分挖掘,此外低照度进一步加剧了跨模态融合的困难。针对以上问题,提出了一种动态亮度重建的无人机可见光-红外融合目标检测方法。首先,利用局部照明信息先验,设计了一个超像素动态照明感知掩膜(Super-Pixel Dynamic Illumination-Aware Mask, SDIM)模块,通过模拟真实场景对不同模态的依赖并引入超像素信息,解决了现有方法存在的物体边缘特征丢失问题。其次,针对低照度可见光图像特征退化问题,设计了低照度图像增强(Low Illumination Image Enhance, LIIE)模块,实现了面向检测任务的端到端优化的可见光图像关键语义自适应增强。最后,针对跨模态特征异构性引发的融合冲突,设计了多尺度特征交叉注意融合(Multi-Scale Feature Cross-Attention Fusion, MFCF)模块,通过层级化交叉注意力机制构建双模态特征交互空间,结合动态权重分配策略,实现了多尺度特征自适应融合。基于典型的可见光-红外融合目标检测数据集DroneVehicle和VEDAI的实验结果,验证了所提方法在低照度条件下可见光-红外融合目标检测任务中的有效性及鲁棒性,具体与现有先进的融合检测算法相比,其保持较低的参数量的同时平均精度(mAP)分别提升了2.3%和2.2%,并且相较于被广泛使用的单模态YOLOv8算法,mAP最高提升了12.9%。此外,基于LIS真实低光数据集的跨场景实验结果,进一步验证了所提方法良好的泛化性。

关键词: 低照度, 无人机, 可见光-红外, 深度学习, 目标检测

Abstract:

The visible-infrared fusion object detection of Unmanned Aerial Vehicles (UAV) has important application value in military and civilian fields such as disaster rescue, security monitoring, and battlefield reconnaissance. However, under low illumination conditions, existing fusion strategies have several limitations, including ignoring uneven lighting in different areas of the same scene and over -reliance on infrared modalities, which results in the potential rich semantic information of visible images in low illumination conditions. In addition, low light further exacerbates the difficulty of cross modal fusion. To address the above problems, a dynamic brightness reconstruction for UAV visible-infrared fusion object detection method is proposed. Firstly, a Super pixel Dynamic Illumination aware Mask (SDIM) module was designed using prior local illumination information. By simulating the dependence of real scenes on different modalities and introducing superpixel information, the problem of object edge feature loss in existing methods was solved. Secondly, considering the problem of feature degradation in low light visible images, a Low Illumination Image Enhancement (LIIE) module was designed to achieve end-to-end optimization of visible image key semantic adaptive enhancement for detection tasks. Finally, a Multi-Scale Feature Cross-Attention Fusion (MFCF) module was designed to address the fusion conflicts caused by cross modal feature heterogeneity. The module constructs a bimodal feature interaction space through a hierarchical cross attention mechanism and adaptively fuses multi-scale features using a dynamic weight allocation strategy. Based on the typical visible-infrared fusion object detection datasets DroneVehicle and VEDAI, the experimental results verified the effectiveness and robustness of the proposed method in visible-infrared fusion target detection tasks under low illumination conditions. Specifically, compared with existing advanced fusion detection algorithms, the proposed method improved the average accuracy (mAP) by 2.3% and 2.2% respectively while maintaining a low number of parameters, and compared with the widely used single-mode YOLOv8 algorithm, the mAP has increased by up to 12.9%. In addition, cross scene experimental results based on the LIS real low illumination dataset further validated the excellent generalization capability of the proposed method.

Key words: low illumination, UAV, visible-infrared, deep learning, object detection

中图分类号: