A Novel Hybrid-Action-Based Deep Reinforcement Learning for Industrial Energy Management

As environmental pollution becomes increasingly serious and industrial energy consumption continuously rises, an intelligent and efficient industrial energy management policy is urgently needed to reduce costs and maximize the benefits of industrial energy systems. However, modern industrial energy systems are characterized by hybrid industrial equipment actions, diverse objectives, and highly intermittent and stochastically distributed renewable energy sources. Therefore, efficient operation and control are difficult. This article presents a novel, model-free energy management policy using a hybrid action deep reinforcement learning algorithm for energy scheduling of industrial equipments operating in various modes. Specifically, the interaction process between the industrial energy management center and each equipment is modeled as a Markov decision process that minimizes the daily operating cost of the energy system and maximizes the revenue of the production equipment. Then, a double parameterized deep Q-networks that does not require an explicit environmental model is developed to learn the hybrid action signals using actor and critic networks, in which the double Q value mechanism avoids value overestimation and improves the algorithm efficiency. In addition, the policy gradient of the proposed algorithm is derived and its convergence proof is discussed. Finally, numerical studies are conducted using real-world data to evaluate algorithm performance and verify its effectiveness.