Improving Accuracy and Efficiency in Time Series Forecasting with an Optimized Transformer Model

Time series forecasting (TSF) is a prevalent research task in various fields such as medicine, transportation, environment, network detection, finance, and others. The TSF task aims to identify underlying patterns in data and make relatively accurate estimates of future data based on known values. In recent years, deep learning models have gained popularity for TSF tasks due to their capability to capture internal information effectively. However, traditional deep-learning models encounter difficulties when parallelizing data calculations, leading to error accumulation and reduced forecasting accuracy. Additionally, when dealing with excessively long input data, traditional deep learning models may experience performance degradation despite providing sufficient information and making it arduous to predict future data. Transformer-based models, with Self-Attention as the core, have shown the ability to facilitate global information interaction and enhance prediction efficiency. Nonetheless, they may encounter problems with significant and redundant parameters, causing unnecessary time overhead. To overcome these challenges, we propose a novel model called VarSeg-Trans, which incorporates three key optimizations: the cut-up mechanism, the variables-isolating mechanism, and an improved attention calculation method to enhance the transformer model's performance. Specifically, the cut-up mechanism enables the model to process longer input sequences, the variables-isolating mechanism mitigates overfitting, and the improved attention method leverages sequence information more effectively. Compared to other baseline TSF models and previous Transformer-based models, VarSeg-Trans has achieved an average reduction of 9% in MSE and MAE, along with a 3% increase in the coefficient of determination R2. This trend is substantiated by consistent results across multiple experimental trials.