On the analysis of adapting deep learning methods to hyperspectral imaging. Use case for WEEE recycling and dataset

Hyperspectral imaging, a rapidly evolving field, has witnessed the ascendancy of deep learning techniques, supplanting classical feature extraction and classification methods in various applications. However, many researchers employ arbitrary architectures for hyperspectral image processing, often without rigorous analysis of the interplay between spectral and spatial information. This oversight neglects the implications of combining these two modalities on model performance, consumption, and inference time. This paper evaluates the impact of including different spatial (visual texture) and spectral (captured spectral information) features on deep learning architectures for hyperspectral image segmentation. To this end, it presents different architectural configurations with varying levels of spectral and spatial information and are evaluated in terms of identification performance, energy consumption, and inference time. Additionally, the transferability of knowledge from large pre-trained image foundation models, originally designed for RGB images, to the hyperspectral domain is explored. Results show that incorporating spatial information alongside spectral data leads to improved segmentation results. However, not all spectral wavelengths are necessary to obtain the optimal performance/energy consumption ratio, which is required for faster and more carbon-neutral models. Training foundation models from the RGB domain leads to lower performance and higher energy consumption models with longer inference times. It is also essential to further develop novel architectures that integrate spectral and spatial information and adapt RGB foundation models to the hyperspectral domain. Furthermore, this paper contributes to the field by cleaning and publicly releasing the Tecnalia WEEE Hyperspectral dataset. This dataset contains different non-ferrous fractions of Waste Electrical and Electronic Equipment (WEEE), including Copper, Brass, Aluminum, Stainless Steel, and White Copper, spanning the range of 400 to 1000 nm