HQ-DTM: A Hierarchical Q-learning Algorithm for Dynamic Thermal Management of Multi-core Processors

In this paper, we propose a hierarchical Q-learning algorithm for dynamic thermal management (HQ-DTM) of multi-core processors. The proposed technique aims to maximize the performance of the processor subject to temperature constraints, while minimizing temperature measurement overhead. In order to achieve this, a Q-learning-based temperature measurement module is deployed alongside a Q-learning-based working mode selection module. The temperature measurement module determines the granularity of thermal model to be used under different workloads to ensure that temperature measurement overhead remains minimal while satisfying temperature constraints. The temperature measured is provided as input to the working mode selection module. It uses dynamic voltage and frequency scaling as control to maximize the throughput of each core while maintaining temperature and temporal temperature gradient (TTG) constraints. Our proposed technique has been evaluated for the SPLASH2 benchmarks. The results shows clear evidence that HQ-DTM mitigates the trade-off between processor performance and maintaining optimum thermal behavior.