导航

ACTA AERONAUTICAET ASTRONAUTICA SINICA ›› 2022, Vol. 43 ›› Issue (S1): 726938.doi: 10.7527/S1000-6893.2022.26938

• Information Fusion • Previous Articles     Next Articles

Infrared and visible image fusion based on unsupervised deep learning

SUN Xiuyi1,2, HU Shaohai1,2, MA Xiaole1,2   

  1. 1. Institute of Information Science, Beijing Jiaotong University, Beijing 100044, China;
    2. Beijing Key Laboratory of Advanced Information Science and Network Technology, Beijing 100044, China
  • Received:2022-01-11 Revised:2022-01-14 Published:2022-04-06
  • Supported by:
    National Natural Science Foundation of China (62172030, 61771058), the Fundamental Research Funds for the Central Universities (2021JBM009)

Abstract: Most of the known infrared and visible image fusion models based on convolutional neural networks make little use of the hierarchical features from visible images, thus resulting in insufficient texture details of the fused image. Inspired by the residual network and dense network, an image fusion algorithm is proposed based on unsupervised deep learning to solve the problem of insufficient texture information of fused images. The residual dense block has a continuous storage mechanism to retain the feature information of each layer to the maximum extent. The design of local residual fusion and global residual fusion is conducive to learning the structural texture in the image. In addition, to better preserve the detailed texture in visible images, the generative adversarial network is introduced to perform unsupervised learning on the dataset. Subjective and objective experiments show that the proposed algorithm achieves not only a good visual fusion effect, but also more edge texture information of the fused image. Compared with that of the existing state-of-the-art algorithms, the objective evaluation index of the method proposed is also greatly improved.

Key words: unsupervised learning, residual dense network, generative adversarial network, infrared image, visible image, image fusion

CLC Number: