On Learning Adaptive Service Compositions

Service composition is an important and effective technique that enables atomic services to be combined together to forma more powerful service, i.e., a composite service. With the pervasiveness of the Internet and the proliferation of interconnected computing devices, it is essential that service composition embraces an adaptive service provisioning perspective. Reinforcement learning has emerged as a powerful tool to compose and adapt Web services in open and dynamic environments. However, the most common applications of reinforcement learning algorithms are relatively inefficient in their use of the interaction experience data, whichmay affect the stability of the learning process when deployed to cloud environments. In particular, they make just one learning update for each interaction experience. This paper introduces a novel approach that aims to achieve greater data efficiency by saving the experience data and using it in aggregate to make updates to the learned policy. The proposed approach devises an offline learning scheme for cloud service composition where the online learning task is transformed into a series of supervised learning tasks. A set of algorithms is proposed under this scheme in order to facilitate and empower efficient service composition in the cloud under various policies and different scenarios. The results of our experiments show the effectiveness of the proposed approach for composing and adapting cloud services, especially under dynamic environment settings, compared to their online learning counterparts.