Multilevel parallel attention knowledge distillation for multimodal biometric recognition

Multimodal Biometric Recognition (MBR) is a biometric technology designed to meet stringent security requirements in high-stakes scenarios. During the last decade, researchers have proposed various approaches to improve MBR, such as combining redundant biometric features and designing sophisticated recognition networks. However, the increasing complexity of these networks and the integration of additional features have substantially increased computational demands and prolonged training cycles, thereby hindering deployment on resource-constrained embedded systems. To address the aforementioned computational and deployment constraints, this paper proposes a Multilevel Parallel Attention Knowledge Distillation (MPAD) framework to compress MBR models. Unlike traditional knowledge distillation frameworks that only transfer unimodal features, our approach transfers attention activation maps from a complex teacher network to a lightweight student network through hierarchical layer alignment. Specifically, instead of guiding the student network's learning of each modality with isolated teacher's features, MPAD implements multilevel parallel attention fusion by combining spatial and channel attention mechanisms. This dual-attention design enables the model to simultaneously prioritize critical spatial regions and channel-wise dependencies, thereby enhancing the representational fidelity of distilled knowledge. Furthermore, a linear attention module generates batch-level activation matrices, which enforce similarity constraints between the feature extraction processes of the student and teacher networks. Extensive experiments on two real-world multimodal biometric datasets demonstrate that student models trained with our approach outperform those trained with other advanced knowledge distillation methods.