Scalable and Locality-Aware Distributed Topic-based Pub/Sub Messaging for IoT

Topic-based pub/sub (TBPS) messaging plays an important role in building event-driven Internet of Things (IoT) applications. In IoT applications, scalability and locality-awareness are important properties that help to achieve low-latency message delivery and efficient usage of network resources. However, none of the existing distributed TBPS methods can simultaneously achieve a sufficient level of both properties. This paper proposes a new TBPS overlay method called 'Skip Graph-based TBPS with Locality-Awareness' (STLA), which extends existing Skip Graph-based TBPS messaging by adding locality-awareness. STLA determines the order of the keys on a Skip Graph overlay network according to the network hierarchy structure using 'locality-aware topic keys' (LATK). Using 'split-forward broadcasting' (SFB) with LATK, the locality-awareness can be dramatically improved. Simulation results show that our method can achieve locality-awareness and reduce the average latency of message delivery for 100,000 subscribers by 76% compared with existing methods. In addition, we have conducted experiments on real distributed data centers using an STLA prototype system, and have confirmed the practicality and feasibility of the proposed method.