Learning query optimization method based on multi model outside database

被引:0
作者
Li G.-L. [1 ]
Shen D.-R. [1 ]
Nie T.-Z. [1 ]
Kou Y. [1 ]
机构
[1] School of Computer Science and Engineering, Northeastern University, Shenyang
来源
Zhejiang Daxue Xuebao (Gongxue Ban)/Journal of Zhejiang University (Engineering Science) | 2022年 / 56卷 / 02期
关键词
Cardinality estimation; Join order; Neural network; Query optimization; Reinforcement learning;
D O I
10.3785/j.issn.1008-973X.2022.02.009
中图分类号
学科分类号
摘要
For AI and database optimization problems, existing technologies need to change the bottom layer of database, which affects the application of research results and lacks scalability. A learning query optimization method for non-embedded database was proposed. In the cardinality estimation stage, the multi-model method is used to establish a neural network for specific sub queries and train different sub models independently, which solves the problem of too many training sets and poor scalability. In the join optimization stage, cost-based reinforcement learning is applied to improve the query optimization performance. For each query, the optimization processes from cardinality estimation to connection sorting are executed outside the database. The query is rewritten according to the obtained optimization strategy, and the rewriting results are returned to the database. The query is executed according to the specified plan by setting parameters. Experimental verification was carried out on the data set containing eight tables. Compared with the query not optimized, the optimization method of non-embedded database has good optimization effect. Copyright ©2022 Journal of Zhejiang University (Engineering Science). All rights reserved.
引用
收藏
页码:288 / 296
页数:8
相关论文
共 19 条
[1]  
ARMSTRONG K., Big data: a revolution that will transform how we live, work, and think, Mathematics and Computer Education, 47, 10, pp. 181-183, (2014)
[2]  
LI Guo-liang, ZHOU Xuan-he, SUN Ji, Et al., A survey of machine learning based database techniques, Chinese Journal of Computers, 43, 11, pp. 2019-2049, (2020)
[3]  
DURAND M, FLAJOLET P., Loglog counting of large cardinalities, 11th Annual European Symposium on Algorithms, (2003)
[4]  
FLAJOLET P, FUSY E, GANDOUET O, Et al., Hyperloglog: the analysis of a near-optimal cardinality estimation algorithm, Discrete Mathematics and Theoretical Computer Science, pp. 137-156, (2007)
[5]  
WU C, JINDAL A, AMIZADEH S, Et al., Towards a learning optimizer for shared clouds, Proceedings of the VLDB Endowment, 12, 3, pp. 210-222, (2018)
[6]  
LAKSHMI M S, ZHOU S., Selectivity estimation in extensible databases: a neural network approach, Proceedings of 24th International Conference on Very Large Data Bases, (1998)
[7]  
DUTT A, WANG C, NAZI A, Et al., Selectivity estimation for range predicates using lightweight models, Proceedings of the VLDB Endowment, 12, 9, pp. 1044-1057, (2019)
[8]  
HASAN S, THIRUMURUGANATHAN S, AUGUSTINE J, Et al., Multi-attribute selectivity estimation using deep learning
[9]  
KIPF A, KIPF T, RADKE B, Et al., Learned cardinalities: estimating correlated joins with deep learning
[10]  
SUN, LI G., An end-to-end learning-based cost estimator, Proceedings of the VLDB Endowment, 13, 3, pp. 307-319, (2019)