Decorrelating structure via adapters makes ensemble learning practical for Semi-supervised Learning

In Semi-Supervised Learning (SSL), ensemble learning boosts training by enhancing pseudo-label accuracy through multi-perspective prediction integration. This improvement hinges on ensemble framework diversity. However, balancing ensemble cost and diversity is crucial. High costs can impede practicality or efficiency, while low diversity limits performance gains. Our experiments comparing ensemble methods reveal that diversity is influenced by prediction similarity among predictors. Thus, we propose ”Decorrelating Structure via Adapters” (DSA), using distinct adapters to map input features of each prediction head into independent spaces, fostering diversity. As a lightweight structure, DSA effectively decorrelates without substantially increasing parameters or relying on extra loss functions. We analyze DSA's mechanism, explaining how adapters reduce prediction similarity. Moreover, we showcase DSA's significant improvements in semi-supervised classification and pose estimation tasks. The code is publicly accessible at: https://github.com/Qi2019KB/DSA. © 2025 Elsevier Ltd