IIsy: Hybrid In-Network Classification Using Programmable Switches

The soaring use of machine learning leads to increasing processing demands. As data volume keeps growing, providing classification services with good machine learning performance, high throughput, low latency, and minimal equipment overheads becomes a challenge. Offloading machine learning tasks to network switches can be a scalable solution to this problem, providing high throughput and low latency. However, network devices are resource constrained, and lack support for machine learning functionality. In this paper, we introduce IIsy - a novel mapping tool of machine learning classification models to off-the-shelf switches. Using an efficient encoding algorithm, IIsy enables fitting a range of classification models on switches, co-existing with standard switch functionality. To overcome resource constraints, IIsy adopts a hybrid approach for ensemble models, running a small model on a switch and a large model on the backend. The evaluation shows that IIsy achieves near-optimal classification results, within minimum resource overheads, and while reducing the load on the backend by 70% for data-intensive use cases.