AVAFN-adaptive variational autoencoder fusion network for multispectral image

Image fusion involves extracting and combining the most meaningful information from images captured by different sensors, aiming to create a single image beneficial for further applications. The development of deep learning has significantly advanced image fusion, with neural networks' powerful feature extraction and reconstruction capabilities broadening the prospects for fused images. This study introduces the Adaptive Variational Autoencoder Image Fusion Network (AVAFN), enhancing the Variational Autoencoder (VAE) architecture by incorporating additional convolutional, ReLU, and Dropout layers to improve feature extraction and prevent overfitting. AVAFN introduces a novel loss function that considers global contrast, feature intensity, and structural texture, reducing feature and contrast loss during fusion and preserving detail texture in line with human visual perception. The importance of loss function weights is evident in AVAFN's Adaptive Loss Assignment Method, which adjusts loss weights based on the different feature information of input images, analyzes various depth features, and obtains optimal fusion outcomes. Experimental results reveal that AVAFN excels in multi-modal and multi-exposure image fusion, achieving scene-appropriate fusion results through color channel transformation and effectively extracting depth features in multi-input image fusion tasks through the Concatenation function. These results, confirmed by comparisons with popular fusion methods across three image databases, demonstrate AVAFN's superiority in subjective visual perception and objective fusion metrics. The experiments validate AVAFN's applicability to various image fusion tasks, achieving the desired fusion outcomes even in low-quality images and showcasing its potential in the image fusion domain. © The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2024.