Enterprise and service-level scheduling of robot production services in cloud manufacturing with deep reinforcement learning

Cloud manufacturing is a manufacturing paradigm that integrates wide-area distributed manufacturing resources for distributed services over the Internet. Scheduling is a critical technique that determines the overall performance of a cloud manufacturing system. Robots are an important type of manufacturing resource in cloud manufacturing. Scheduling of robot production services is therefore an important research issue in cloud manufacturing. In cloud manufacturing, services can be selected at an enterprise level or a service level, which represents two types of ways of scheduling. Which way is better and how to select the optimal robot production services are issues that have rarely been considered. Recently, deep reinforcement learning (DRL) has been successfully applied to solving various scheduling problems from different fields. Given this, this paper investigates enterprise and service-level scheduling of robot production services in cloud manufacturing and explores the optimal ways and methods of scheduling with DRL. Deep Q-Networks (DQN) and its three modified algorithms, including Double DQN, Dueling DQN, and Average-DQN based on scheduling approaches are proposed. Effects of enterprise- and service-level robot production services selection methods in cloud manufacturing are studied. Comparative results indicate that overall the service-level selection method outperforms the enterprise-level method. The performance of the above-mentioned scheduling algorithms is further studied with the service-level selection method. Results indicate that the Average-DQN-based approach is able to generate scheduling solutions more efficiently and performs the best with respect to each metric.