Deep Reinforcement Learning Enabled Self-Configurable Networks-on-Chip for High-Performance and Energy-Efficient Computing Systems

Network-on-Chips (NoC) has been the superior interconnect fabric for multi/many-core on-chip systems because of its scalability and parallelism. On-chip network resources can be dynamically configured to improve the energy efficiency and performance of NoC. However, large and complex design space in heterogeneous NoC architectures becomes difficult to explore within a reasonable time for optimal trade-offs of energy and performance. Furthermore, reactive resource management is not effective in preventing problems, such as thermal hotspots, from happening in adaptive systems. Therefore, we propose machine learning (ML) techniques to provide proactive solutions within an instant in NoC-based computing systems. We present a deep reinforcement learning (deep RL) technique to configure voltage/frequency levels of NoC routers and links for both high performance and energy efficiency while meeting the global energy budget constraint. Distributed RL agents technique has been proposed, where an RL agent configures a NoC router and associated links intelligently based on system utilization and application demands. Additionally, neural networks are used to approximate the actions of distributed RL agents. Simulations results for NoC sizes ranging from 16 to 256 cores under real applications and synthetic traffic show that the proposed self-configurable and scalable approach, on average, improves energy-delay product (EDP) by 30-40% (up to 80%) and by 8% (up to 17%) compared to existing non-ML and ML based solutions, respectively.