A Low-Power Circuit for Adaptive Dynamic Programming

This paper presents a low-power CMOS design for accelerating an adaptive dynamic programming algorithm, called action-dependent heuristic dynamic programming, which is widely employed in many real-life control problems. The objective of this work is to solve Bellman equation approximately using efficient hardware in order to generate near-optimal real-time control policies for many control applications. The hardware exploits the data-level parallelism exists in both inference and learning of neural networks in order to improve throughput as well as to provide good scalability. The circuit is realized in a 65-nm technology. It is shown with simulations that the design is two orders of magnitude faster than a software running on a general-purpose processor thanks to the parallelization of the algorithm and the reduction in unnecessary control overheads. Performance of the CMOS circuit is benchmarked with two popular control tasks. Successful learning can be achieved with a power consumption of 28 mW.