Node selection using adversarial expert-based multi-armed bandits in distributed computing

The edge computing (EC) paradigm enhances the Quality of Service of distributed computing applications by bringing computation closer to data sources, such as sensors, IoT devices, and local servers, instead of relying solely on centralized data centers (e.g., the Cloud). In EC environments, node selection refers to the problem of determining which distributed computing nodes should be selected for performing computing tasks taking into consideration the heterogeneity of factors like limited resources, network context, and node's computational capabilities. Evidently, node selection affects the efficiency and performance of EC environments. Recent node selection strategies rely on either heuristic or optimization methods, which inherently assume static environments. However, distributed environments consist of highly heterogeneous and dynamic systems. Addressing such a dynamic nature requires node selection strategies that leverage real-time feedback information. In this paper, we propose sequential learning-based algorithms based on multi-armed bandit (MAB) systems to deal with the node selection problem. Unlike previous MAB approaches, we contribute novel MAB algorithms for node selection using deep learning expert models. To tackle the inherent uncertainty associated with nodes, we introduce ExpGradBand, a novel expert-based gradient MAB algorithm, which leverages the selection efficiency of gradient bandits with the historic contextual information. Furthermore, we evaluate and compare ExpGradBand with various MAB approaches and baselines found in the literature with and without contextual information. Our evaluation study includes comprehensive experiments that assess the performance of these methods in settings with delayed or lost contextual feedback.