EC2Detect: Real-Time Online Video Object Detection in Edge-Cloud Collaborative IoT

Video object detection is a fundamental technology of intelligent video analytics for Internet of Things (IoT) applications. However, even with extraordinary detection accuracy, predominating solutions based on deep convolutional neural networks (DCNNs) cannot achieve real-time online object detection on video streams with a low end-to-end (E2E) response latency and therefore cannot be applied to proliferating latency-sensitive IoT applications like autonomous driving requiring large-scale intelligent video analytics. To address this issue, we present EC(2)Detect, an edge-cloud collaborative real-time online video object detection method. Specifically, we propose a tracking-assisted object detection architecture based on edge-cloud collaboration with keyframe selection, where the accurate but heavy object detection is conducted by the Cloud on sparse keyframes adaptively selected according to their semantic variation, and the lightweight object tracking is used to localize and identify objects in other frames at edge devices. Extensive experiments of our real-world prototype demonstrate that, EC(2)Detect significantly outperforms state-of-the-art methods in terms of processing speed (up to 4.77x faster), E2E latency (up to 8.12x lower), and edge-cloud bandwidth occupation (17x lower) with an acceptable mAP, which can effectively support large-scale intelligent video analytics in practice. Source code of EC Detect is available at https://github.com/ECCDetect/ECCDetect.