Most of the known infrared and visible image fusion models based on convolutional neural networks make little use of the hierarchical features from visible images， thus resulting in insufficient texture details of the fused image. Inspired by the residual network and dense network， an image fusion algorithm is proposed based on unsupervised deep learning to solve the problem of insufficient texture information of fused images. The residual dense block has a continuous storage mechanism to retain the feature information of each layer to the maximum extent. The design of local residual fusion and global residual fusion is conducive to learning the structural texture in the image. In addition， to better preserve the detailed texture in visible images， the generative adversarial network is introduced to perform unsupervised learning on the dataset. Subjective and objective experiments show that the proposed algorithm achieves not only a good visual fusion effect， but also more edge texture information of the fused image. Compared with that of the existing state-of-the-art algorithms， the objective evaluation index of the method proposed is also greatly improved.
SUN Xiuyi
,
HU Shaohai
,
MA Xiaole
. Infrared and visible image fusion based on unsupervised deep learning[J]. ACTA AERONAUTICAET ASTRONAUTICA SINICA, 2022
, 43(S1)
: 726938
-726938
.
DOI: 10.7527/S1000-6893.2022.26938
[1] SHEN Y, HUANG C H, HUANG F, et al. Research progress of infrared and visible image fusion technology[J]. Infrared and Laser Engineering, 2021, 50(9): 152-169 (in Chinese). 沈英, 黄春红, 黄峰, 等. 红外与可见光图像融合技术的研究进展[J]. 红外与激光工程, 2021, 50(9): 152-169.
[2] CHEN J, WU K L, CHENG Z, et al. A saliency-based multiscale approach for infrared and visible image fusion[J]. Signal Processing, 2021, 182: 107936.
[3] YANG Y, LIU J X, HUANG S Y, et al. Infrared and visible image fusion via texture conditional generative adversarial network[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31(12): 4771-4783.
[4] BAI Y, HOU Z Q, LIU X Y, et al. An object detection algorithm based on decision-level fusion of visible light image and infrared image[J]. Journal of Air Force Engineering University (Natural Science Edition), 2020, 21(6): 53-59, 100 (in Chinese). 白玉, 侯志强, 刘晓义, 等. 基于可见光图像和红外图像决策级融合的目标检测算法[J]. 空军工程大学学报(自然科学版), 2020, 21(6): 53-59, 100.
[5] ZHANG H, XU H, XIAO Y, et al. Rethinking the image fusion: A fast unified image fusion network based on proportional maintenance of gradient and intensity[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(7): 12797-12804.
[6] LIU Y, CHEN X, PENG H, et al. Multi-focus image fusion with a deep convolutional neural network[J]. Information Fusion, 2017, 36: 191-207.
[7]
[8]
[9]
[10] LI H, WU X J. DenseFuse: A fusion approach to infrared and visible images[J]. IEEE Transactions on Image Processing, 2018: 2018Dec18.
[11]
[12] RUDIN L I, OSHER S, FATEMI E. Nonlinear total variation based noise removal algorithms[J]. Physica D: Nonlinear Phenomena, 1992, 60(1-4): 259-268.
[13]
[14] TOET A. Image fusion by a ratio of low-pass pyramid[J]. Pattern Recognition Letters, 1989, 9(4): 245-253.
[15] SHREYAMSHA KUMAR B K. Image fusion based on pixel significance using cross bilateral filter[J]. Signal, Image and Video Processing, 2015, 9(5): 1193-1204.
[16] LIU Y, LIU S P, WANG Z F. A general framework for image fusion based on multi-scale transform and sparse representation[J]. Information Fusion, 2015, 24: 147-164.
[17] MA J Y, CHEN C, LI C, et al. Infrared and visible image fusion via gradient transfer and total variation minimization[J]. Information Fusion, 2016, 31: 100-109.
[18] MA J Y, YU W, LIANG P W, et al. FusionGAN: A generative adversarial network for infrared and visible image fusion[J]. Information Fusion, 2019, 48: 11-26.
[19] MA J Y, XU H, JIANG J J, et al. DDcGAN: A dual-discriminator conditional generative adversarial network for multi-resolution image fusion[J]. IEEE Transactions on Image Processing, 2020, 29: 4980-4995.
[20] MA J Y, ZHANG H, SHAO Z F, et al. GANMcC: A generative adversarial network with multiclassification constraints for infrared and visible image fusion[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-14.
[21]
CJA