Geometry-Adaptive Meta-Learning in Mixed-Curvature Spaces

被引:0
作者
Gao, Zhi [1 ]
Wu, Yu-Wei [1 ,2 ]
Jia, Yun-De [1 ,2 ]
机构
[1] Beijing Key Laboratory of Intelligent Information Technology, School of Computer Science & Technology, Beijing Institute of Technology, Beijing
[2] Guangdong Laboratory of Machine Perception and Intelligent Computing, Shenzhen MSU-BIT University, Guangdong, Shenzhen
来源
Jisuanji Xuebao/Chinese Journal of Computers | 2024年 / 47卷 / 10期
基金
中国国家自然科学基金;
关键词
geometry adaptation; meta-learning; mixed-curvature space; Riemannian manifold;
D O I
10.11897/SP.J.1016.2024.02289
中图分类号
学科分类号
摘要
Meta-learning has shown effectiveness in helping learning models quickly adapt to new tasks by learning prior knowledge. In the process of adaptation to new tasks, the matching degree between the geometric structure of space and the geometric structure of data plays an important role in the generalization ability of the model. In many practical applications, data has diverse non-Euclidean structures. For example, natural language has non-Euclidean hierarchical structures, and face images have non-Euclidean cyclical structures. Existing research has shown that the geometric structure of Riemannian manifolds matches the non-Euclidean structures of real-world data, providing theoretical feasibility for modeling data using Riemannian manifolds. In this paper, we propose a geometry-adaptive meta-learning method in mixed-curvature spaces, which uses multiple mixed-curvature spaces to model data and produces matching Riemannian geometry for non-Euclidean structures. We build a multi-mixed-curvature neural network that represents the geometry of mixed-curvature space as curvature, number, and dimensionality of the curvature spaces, through which the geometry adaptation to non-Euclidean structures is achieved via a gradient descent process. We further introduce a geometry initialization generation scheme and geometry updating scheme. Through only a few optimization steps, the geometric structure of the underlying space can quickly match non-Euclidean structures of data, accelerating the gradient descent process. We conduct experiments on few-shot classification, few-shot regression, and image completion to evaluate the effectiveness of our method. Compared with meta-learning methods in Euclidean space, our method improves the accuracy by 3% in few-shot classification tasks, and reduces mean square error by half in few-shot regression tasks, showing the effectiveness of our method. © 2024 Science Press. All rights reserved.
引用
收藏
页码:2289 / 2306
页数:17
相关论文
共 53 条
  • [51] Collins L, Mokhtari A, Shakkottai S., Task-robust model-agnostic meta-learning, Advances in Neural Information Processing Systems, pp. 18860-18871, (2020)
  • [52] Garnelo M, Rosenbaum D, Maddison C, Et al., Conditional neural processes, Proceedings of the International Conference on Machine Learning, pp. 1704-1713, (2018)
  • [53] Kruskal J., Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis, Psychometrika, 29, pp. 1-27, (1964)