Augmenting Practical Cross-layer MAC Schedulers via Offline Reinforcement Learning

An automated offline design process for optimized cross-layer schedulers can produce augmented scheduling algorithms tailored to a target deployment scenario. We discuss the application of ODDS, a reinforcement learning technique we introduced, for augmenting LTE MAC scheduler algorithms of practical significance. ODDS observes the correlation between the value of a utility function and the cross-layer state parameters seen by an instrumented baseline scheduler, determining the best actions via an offline Monte Carlo exploration of the problem space. The result of the ODDS process is a compact definition of a scheduling policy that has been optimized for a target scenario and utility function. In this paper we instrument a production scheduler definition to evaluate the potential of augmented schedulers in practical use, and experiment with awareness to application traffic properties by using a multi-class utility function, yielding scheduling policies that behave differently depending on the features of individual traffic classes.