PTMB: An online satellite task scheduling framework based on pre-trained Markov decision process for multi-task scenario

The ever-growing scale of satellites and the increasing demand for Earth observation have led to significant interest in the problem of online multi -task scheduling for Earth observation satellites. Presently, task scheduling mostly relies on meta-heuristic and reinforcement learning algorithms. However, meta-heuristic algorithms possess slow convergence speed and are not suitable for multi -task scheduling scenarios, while reinforcement learning algorithms are unable to ensure solution quality due to unstable environmental states. In this study, we propose an innovative online micro -batch scheduling framework based on pre-trained reinforcement learning model (PTMB). This framework splits the satellite scheduling issue into two phases: task decision-making and task allocation. We leverage the micro -batch processing mode and introduce a pretrained Markov decision model during the task decision-making phase. Additionally, we incorporate resource pre -allocation, task sequencing, task order shuffle, and other strategies to enhance the overall solution quality. Simulation experiments reveal a significant enhancement in performance of our proposed method. Specifically, when dealing with task sizes of 100 and 300, the task scheduling reward of the proposed framework surpasses that of the improved genetic algorithm by 11.5% and 0.4%, respectively, while reducing time consumption by 97.8% and 99.3%. Furthermore, our framework surpasses online reinforcement learning scheduling method, which is also based on Markov decision process model, achieving improvements of 55.1%, 5.6%, and 15.6% in task scheduling reward and reductions in time consumed by 94.9%, 96.7%, and 99.1% at task sizes of 100, 300, and 500, respectively.