Intelligent and Reconfigurable Architecture for KL Divergence-Based Multi-Armed Bandit Algorithms

The Multi-armed bandit (MAB) algorithms do not need any training phase and can be deployed directly in an unknown environment. MAB algorithms can identify the best arm among several arms by achieving a balance between exploration of all arms and exploitation of optimal arm. The Kullback-Leibler divergence based upper confidence bound (KLUCB) is the state-of-the-art MAB algorithm that optimizes exploration-exploitation trade-off but it is complex due to underlining optimization routine. This limits its usefulness for robotics and radio applications which demand integration of KLUCB with the physical layer on the system on chip (SoC). In this brief, we efficiently map the KLUCB algorithm on SoC by realizing optimization routine via alternative synthesizable computation without compromising on the performance. The proposed architecture is dynamically reconfigurable such that the number of arms, as well as type of algorithm, can be changed on-the-fly. Specifically, after initial learning, on-the-fly switch to light-weight UCB offers around 10-factor improvement in latency and throughput. Since learning duration depends on the unknown arm statistics, an intelligence is embedded in architecture to decide the switching instant. We validate the functional correctness and usefulness of the proposed work via a realistic wireless application and detailed complexity analysis demonstrates its feasibility in realizing intelligent radios.