Hyperspectral and Multispectral Image Fusion Using 3D Wavelet Transforms

• Eduardo Rittner Coelho UNICAMP
• Ana Clara C. Silveira UNICAMP
• Tarik P. e Sá UNICAMP
• Diedre S. do Carmo UNICAMP
• Paula D. P. Costa UNICAMP
• Letícia Rittner UNICAMP
• Denis Gustavo Fantinato UNICAMP

DOI: https://doi.org/10.5753/sibgrapi.est.2024.31665

Resumo

The fusion of multispectral (MSI) and hyperspectral (HSI) images is a crucial technique in various fields such as remote sensing, medical imaging, and agricultural monitoring. MSI captures light across several specific spectral bands, while HSI provides detailed spectral information across contiguous bands. Combining these two types of images leverages the high spatial resolution of MSI and the rich spectral content of HSI, creating a single, high-resolution image that is both spatially and spectrally informative. Traditional wavelet based fusion methods often employ a single wavelet across all dimensions, which can result in suboptimal outcomes due to the different characteristics of spatial and spectral data. This paper explores the use of 3D wavelet transforms with varied wavelets across dimensions to improve the fusion process. Experiments conducted on the ICASSP HyperSkin Challenge dataset showed that a combination of Daubechies on the spatial dimensions and Coiflets on the spectral dimension obtained higher fidelity and SSIM when compared to simpler fusion methods.

Referências

J. M. Amigo, “Chapter 1.1 - Hyperspectral and multispectral imaging: Setting the cene,” in Hyperspectral Imaging, ser. Data Handling in Science and Technology, J. M. Amigo, Ed. Elsevier, 2019, vol. 32, pp. 3–16.

J. Gonzalez-Ibarzabal, M. Franquesa, A. Rodriguez-Montellano, and A. Bastarrika, “Sentinel-2 reference fire perimeters for the assessment of burned area products over latin america and the caribbean for the year 2019,” Remote Sensing, vol. 16, no. 7, 2024.

G. Lu and B. Fei, “Medical hyperspectral imaging: a review,” Journal of Biomedical Optics, vol. 19, no. 1, p. 010901, Jan. 2014.

F. Liu, R. Yang, R. Chen, M. Guindo, Y. He, J. Zhou, X. Lu, M. Chen, Y. Yang, and W. Kong, “Digital techniques and trends for seed phenotyping using optical sensors,” Journal of Advanced Research, 2023.

J. Y. H. Raju Shrestha, “Evaluation and comparison of multispectral imaging systems,” in 22nd Color Imag. Conf., 2014, pp. 107–112.

S. Zhang, X. Kang, Y. Mo, and S. Li, “Noise analysis of hyperspectral images captured by different sensors,” in 2020 IEEE Int. Geoscience and Remote Sensing Symposium (IGARSS 2020), 2020, pp. 2707–2710.

R. Dian, S. Li, B. Sun, and A. Guo, “Recent advances and new guidelines on hyperspectral and multispectral image fusion,” Information Fusion, vol. 69, pp. 40–51, 2021.

K. Amolins, Y. Zhang, and P. Dare, “Wavelet based image fusion techniques — an introduction, review and comparison,” ISPRS Journal of Photog. and Remote Sensing, vol. 62, no. 4, pp. 249–263, 2007.

Y. Zhang and M. He, “3d wavelet transform and its application in multi-spectral and hyperspectral image fusion,” in 2009 4th IEEE Conference on Industrial Electronics and Applications, 2009, pp. 3643–3647.

L. Li and H. Ma, “Pulse coupled neural network-based multimodal medical image fusion via guided filtering and wseml in nsct domain,” Entropy, vol. 23, p. 591, 05 2021.

S. Karim, G. Tong, J. Li, A. Qadir, U. Farooq, and Y. Yu, “Current advances and future perspectives of image fusion: A comprehensive review,” Information Fusion, vol. 90, pp. 185–217, 2023.

C. K. Chui, An introduction to wavelets. Elsevier, 2014. [13] M. Srivastava, Y. Yashu, S. K. Singh, and P. K. Panigrahi, “Multisegmentation through wavelets: Comparing the efficacy of daubechies vs coiflets,” arXiv preprint arXiv:1207.5007, 2012.

B. Arad, R. Timofte, R. Yahel, N. Morag, A. Bernat, Y. Cai, and et al., “NTIRE 2022 spectral recovery challenge and data set,” in 2022 IEEE Conf. Comp. Vis. Patt. Recog. Workshops (CVPRW), 2022, pp. 862–880.

Z. Wang and A. C. Bovik, “Mean squared error: Love it or leave it? a new look at signal fidelity measures,” IEEE Signal Processing Magazine, vol. 26, no. 1, pp. 98–117, 2009.

A. C. C. Silveira, D. S. do Carmo, L. H. Ueda, D. G. Fantinato, P. D. P. Costa, and L. Rittner, “Vision transformer MST++: Efficient hyperspectral skin reconstruction,” in IEEE ICASSP 2024, 2024.