Pushing Intelligence to the Edge with a Stream Processing Architecture

The cloud computing paradigm underpins the Internet of Things (IoT) by offering a seemingly infinite pool of resources for processing/storing extreme amounts of data generated by complex IoT systems. The cloud has established a convenient and widely adopted approach, where raw data are vertically offloaded to cloud servers from resource-constrained edge devices, which are only seen as simple data generators, not capable of performing more sophisticated processing activities. However, there are more and more emerging scenarios, where the amount of data to be transferred over the network to the cloud is associated with increased network latency, making the results of the computation obsolete. As various categories of edge devices are becoming more and more powerful in terms of hardware resources specifically, CPU and memory the established way of off-loading computation to the cloud is not always seen as the most convenient approach. Accordingly, this paper presents a Stream Processing architecture for spreading workload among a local cluster of edge devices to process data in parallel, thus achieving faster execution and response times. The experimental results suggest that such a distributed in-memory approach to data processing at the very edge of a computational network has a potential to address a wide range of IoT-related scenarios.