An adaptive deep-learning load forecasting framework by integrating transformer and domain knowledge

被引:33
作者
Gao, Jiaxin [1 ,2 ]
Chen, Yuntian [1 ,2 ]
Hu, Wenbo [3 ]
Zhang, Dongxiao [1 ,4 ,5 ]
机构
[1] Eastern Inst Technol, Eastern Inst Adv Study, Ningbo, Zhejiang, Peoples R China
[2] Shanghai Jiao Tong Univ, Sch Elect Informat & Elect Engn, Shanghai, Peoples R China
[3] Hefei Univ Technol, Sch Comp & Informat, Hefei, Peoples R China
[4] Peng Cheng Lab, Dept Math & Theories, Shenzhen, Guangdong, Peoples R China
[5] Southern Univ Sci & Technol, Natl Ctr Appl Math Shenzhen NCAMS, Shenzhen, Guangdong, Peoples R China
来源
ADVANCES IN APPLIED ENERGY | 2023年 / 10卷
基金
中国国家自然科学基金;
关键词
Load forecasting; Deep-learning; Domain knowledge; Transfer learning; Online learning; Interpretability; MODELS;
D O I
10.1016/j.adapen.2023.100142
中图分类号
TE [石油、天然气工业]; TK [能源与动力工程];
学科分类号
0807 ; 0820 ;
摘要
Electrical energy is essential in today's society. Accurate electrical load forecasting is beneficial for better schedul-ing of electricity generation and saving electrical energy. In this paper, we propose an adaptive deep-learning load forecasting framework by integrating Transformer and domain knowledge (Adaptive-TgDLF). Adaptive-TgDLF introduces the deep-learning model Transformer and adaptive learning methods (including transfer learning for different locations and online learning for different time periods), which captures the long-term dependency of the load series, and is more appropriate for realistic scenarios with scarce samples and variable data distributions. Under the theory-guided framework, the electrical load is divided into dimensionless trends and local fluctua-tions. The dimensionless trends are considered as the inherent pattern of the load, and the local fluctuations are considered to be determined by the external driving forces. Adaptive learning can cope with the change of load in location and time, and can make full use of load data at different locations and times to train a more efficient model. Cross-validation experiments on different districts show that Adaptive-TgDLF is approximately 16% more accurate than the previous TgDLF model and saves more than half of the training time. Adaptive-TgDLF with 50% weather noise has the same accuracy as the previous TgDLF model without noise, which proves its robustness. We also preliminarily mine the interpretability of Transformer in Adaptive-TgDLF, which may provide future potential for better theory guidance. Furthermore, experiments demonstrate that transfer learning can accelerate convergence of the model in half the number of training epochs and achieve better performance, and online learning enables the model to achieve better results on the changing load.
引用
收藏
页数:15
相关论文
共 50 条
[41]   Using deep learning for short-term load forecasting [J].
Bendaoud, Nadjib Mohamed Mehdi ;
Farah, Nadir .
NEURAL COMPUTING & APPLICATIONS, 2020, 32 (18) :15029-15041
[42]   Adaptive individual residential load forecasting based on deep learning and dynamic mirror descent [J].
Han, Fujia ;
Wang, Xiaohui .
FRONTIERS IN ENERGY RESEARCH, 2023, 10
[43]   A Deep Learning Based Fault Diagnosis Method Combining Domain Knowledge and Transfer Learning [J].
Choudhury, Madhurjya Dev ;
Kleijn, W. Bastiaan ;
Blincoe, Kelly ;
Dhupia, Jaspreet Singh .
2023 29TH INTERNATIONAL CONFERENCE ON MECHATRONICS AND MACHINE VISION IN PRACTICE, M2VIP 2023, 2023,
[44]   KansformerEPI: a deep learning framework integrating KAN and transformer for predicting enhancer-promoter interactions [J].
Zhang, Tianjiao ;
Shao, Saihong ;
Zhang, Hongfei ;
Zhao, Zhongqian ;
Zhao, Xingjie ;
Zhang, Xiang ;
Wang, Zhenxing ;
Wang, Guohua .
BRIEFINGS IN BIOINFORMATICS, 2025, 26 (03)
[45]   Automated deep-learning model optimization framework for microcontrollers [J].
Hong, Seungtae ;
Park, Gunju ;
Kim, Jeong-Si .
ETRI JOURNAL, 2025, 47 (02) :179-192
[46]   A hybrid deep learning framework for short-term load forecasting with improved data cleansing and preprocessing techniques [J].
Iqbal, Muhammad Sajid ;
Adnan, Muhammad ;
Mohamed, Salah Eldeen Gasim ;
Tariq, Muhammad .
RESULTS IN ENGINEERING, 2024, 24
[47]   PDRNet: A Deep-Learning Pedestrian Dead Reckoning Framework [J].
Asraf, Omri ;
Shama, Firas ;
Klein, Itzik .
IEEE SENSORS JOURNAL, 2022, 22 (06) :4932-4939
[48]   Machine Learning and Deep Learning Techniques for Residential Load Forecasting: A Comparative Analysis [J].
Shabbir, Noman ;
Kutt, Lauri ;
Raja, Hadi A. ;
Ahmadiahangar, Roya ;
Rosin, Argo ;
Husev, Oleksandr .
2021 IEEE 62ND INTERNATIONAL SCIENTIFIC CONFERENCE ON POWER AND ELECTRICAL ENGINEERING OF RIGA TECHNICAL UNIVERSITY (RTUCON), 2021,
[49]   Deep-Learning Image Stabilization for Adaptive Optics Ophthalmoscopy [J].
Liu, Shudong ;
Ji, Zhenghao ;
He, Yi ;
Lu, Jing ;
Lan, Gongpu ;
Cong, Jia ;
Xu, Xiaoyu ;
Gu, Boyu .
INFORMATION, 2022, 13 (11)
[50]   A Survey on Deep Learning for Building Load Forecasting [J].
Patsakos, Ioannis ;
Vrochidou, Eleni ;
Papakostas, George A. .
MATHEMATICAL PROBLEMS IN ENGINEERING, 2022, 2022