Value Decomposition based Multi-Task Multi-Agent Deep Reinforcement Learning in Vehicular Networks

With the development of intelligent transportation system (ITS), a multitude of novel vehicular applications have been emerging. There is an urgent need for simultaneously supporting multi-tasks across a group of vehicles in a vehicular network, forming a typical multi-task multi-agent (MTMA) environment. Deep Reinforcement Learning (DRL) is deemed a promising approach to solving the highly complicated MTMA problem. However, owing to the extraordinarily growing computational complexity as well as the explosively increasing dimension of state and action spaces in the MTMA environment, the value functions in the DRL are usually bulky and could be difficult to be learned efficiently. In this way, by virtue of the correlations among multiple vehicular tasks, we adopt the value-decomposition mechanism (VDM) to decompose the complicated value function into several small pieces and then compute each sub-function separately. The proposed paradigm can yield great speed-up in learning and help substantially with a smaller state and action space but without degrading the performance. In this work, we consider an MTMA environment with three vehicular tasks to demonstrate the effectiveness of the proposed mechanism with simulation results.