PrivacyEye: A Privacy-Preserving and Computationally Efficient Deep Learning-Based Mobile Video Analytics System

Large volumes of video data recorded by the increasing mobile devices and embedded sensors can be leveraged to answer queries of our lives, physical world and our evolving society. Especially, the rapid development of convolutional neural networks (CNNs) in the past few years offers the great advantage for multiple tasks in video analysis. However, adopting running CNNs directly on mobile devices and embedded sensors for video analytics brings heavy burden due to their limited capacity, especially for learning a large volume of data. A promising approach is to outsource the computation-intensive part of CNN to cloud. However, the reveal of data to cloud may cause privacy leakage. In addition, the cloud-assisted approach may also bring some communication efficiency challenges for large volume of data. To address both privacy and efficiency issues, we design a privacy-preserving and computationally efficient framework for mobile video analytics. To protect the private information, we split the CNN model into two subnetworks, and first part is used as a feature extractor deployed in the mobile side and the second part is utilized as a classifier deployed in the cloud side. A specific-designed adversarial training process is adopted in order to extract features for normal task classification while hiding the features for sensitive task. In addition, to improve video process efficiency, we design a two-stage framework. The first stage is to extract key frames and necessary intermediate frames, while skipping redundant ones. The second stage is to extract the features of key frames by CNN-based feature extractor but apply optical-flow-based feature propagation algorithm to obtain the features of intermediate frames. Extensive experiments demonstrate our proposed system PrivacyEye can effectively protect private information while keep the accuracy of the normal tasks with less than 2 percent drop, and it saves up to 82.9 percent execution time and 78.8 percent energy consumption.