A new resource-constrained project scheduling problem with ladder-type carbon trading prices and its algorithm based on deep reinforcement learning

Carbon trading aims to reduce emissions through markets and has been adopted by many countries. This study proposes a new resource-constrained project scheduling problem with ladder-type carbon trading prices (RCPSPLCTP). The objective is to minimize the total cost, including the carbon trading cost. We develop a two-stage algorithm called MDDQN-TS to solve the RCPSP-LCTP. First, a multi-step double deep Q-network (MDDQN) with a modified convolutional neural network is trained on small-sized instances to learn the optimal scheduling policy. The learned policy is used to solve instances of various sizes. Second, a tabu search (TS) algorithm is used to further improve the solution obtained by the policy. Experimental results show that MDDQN-TS outperforms both the genetic algorithm (GA) and TS, particularly on large-sized instances. In terms of convergence speed, the MDDQN-TS algorithm demonstrates the fastest performance, followed by the TS algorithm, while GA exhibits the slowest convergence. Specifically, the number of schedules required for MDDQN-TS to converge is only 20.3 % similar to 37.9 % of TS. The experimental results also prove that ladder-type carbon trading prices can reduce carbon emissions more effectively than fixed prices.