Hardware implementation of the upper confidence-bound algorithm for reinforcement learning

The upper confidence-bound algorithm has been identified as a popular and useful approach in reinforcement learning, suitable for solving diverse modern-day problems. In this paper, we have developed efficient, multiple-clock-cycle hardware for this algorithm to ensure its practical application in real-time. The real-life situation that belongs to a class of problems commonly known as multi-armed bandit problems has been observed. The developed design is tested and verified by a field-programmable gate array circuit design. The obtained results have the degree of accuracy of the ones achieved in software simulation, which proofs the robustness of the developed solution. In terms of execution time, the proposed hardware implementation signifi-cantly outperforms the software simulation. Finally, the calculation complexity of the imple-mentation does not depend on the number of observed iterations, which guarantees the effective implementation of the developed design. All implementation details have been provided.