Rethinking Domain Generalization: Discriminability and Generalizability

被引:2
作者
Long, Shaocong [1 ]
Zhou, Qianyu [1 ]
Ying, Chenhao [1 ,2 ]
Ma, Lizhuang [1 ]
Luo, Yuan [1 ,2 ]
机构
[1] Shanghai Jiao Tong Univ, Dept Comp Sci & Engn, Shanghai 200240, Peoples R China
[2] Shanghai Jiao Tong Univ, Blockchain Adv Res Ctr, Wuxi 214101, Jiangsu, Peoples R China
关键词
Domain generalization; representation learning; discriminability; generalizability; transfer learning;
D O I
10.1109/TCSVT.2024.3422887
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Domain generalization (DG) endeavours to develop robust models that possess strong generalizability while preserving excellent discriminability. Nonetheless, pivotal DG techniques tend to improve the feature generalizability by learning domain-invariant representations, inadvertently overlooking the feature discriminability. On the one hand, the simultaneous attainment of generalizability and discriminability of features presents a complex challenge, often entailing inherent contradictions. This challenge becomes particularly pronounced when domain-invariant features manifest reduced discriminability owing to the inclusion of unstable factors, i.e., spurious correlations. On the other hand, prevailing domain-invariant methods can be categorized as category-level alignment, susceptible to discarding indispensable features possessing substantial generalizability and narrowing intra-class variations. To surmount these obstacles, we rethink DG from a new perspective that concurrently imbues features with formidable discriminability and robust generalizability, and present a novel framework, namely, Discriminative Microscopic Distribution Alignment (DMDA). DMDA incorporates two core components: Selective Channel Pruning (SCP) and Micro-level Distribution Alignment (MDA). Concretely, SCP attempts to curtail redundancy within neural networks, prioritizing stable attributes conducive to accurate classification. This approach alleviates the adverse effect of spurious domain-invariance and amplifies the feature discriminability. Besides, MDA accentuates micro-level alignment within each class, going beyond mere category-level alignment. This strategy accommodates sufficient generalizable features and facilitates within-class variations. Extensive experiments on four benchmark datasets corroborate that DMDA achieves comparable results to state-of-the-art methods in DG, underscoring the efficacy of our method. The source code will be available at https://github.com/longshaocong/DMDA.
引用
收藏
页码:11783 / 11797
页数:15
相关论文
共 93 条
  • [91] Van Der Maaten L., Hinton G., Visualizing data using t-SNE, J. Mach. Learn. Res., 9, pp. 2579-2605, (2008)
  • [92] Srivastava N., Hinton G., Krizhevsky A., Sutskever I., Salakhutdinov R., Dropout: A simple way to prevent neural networks from overfitting, J. Mach. Learn. Res., 15, 1, pp. 1929-1958, (2014)
  • [93] Ben-David S., Blitzer J., Crammer K., Kulesza A., Pereira F., Vaughan J.W., A theory of learning from different domains, Mach. Learn., 79, 1, pp. 151-175, (2010)