A Lightweight Heatmap-based Eye Tracking System

Eye tracking is playing an important role in many applications including human-computer interaction and behavior study. However, the existing approaches have at least one of the following limitations: (i) dedicated devices such as infrared camera and eye-tracker are required; (ii) complex calibration process is involved; (iii) substantial computing resources are consumed; (iv) users suffer from the risk of privacy leakage. To address the above limitations, we propose a Heatmap-based Eye Tracking (HETrack) system. One of the key challenges in our system is to design a lightweight model for fine-grained tracking when the computing resources of device is limited. Also, it is necessary to protect user privacy in such a system. To address the above challenging issues, the proposed system consists of the following processes. First, when users randomly look at the screen of the device, HETrack obtains the raw image containing facial information. Then, we design a neural network model and train it with federated learning. The model can map the image to heatmap that implies the possibility of the user's gaze position on the screen. Finally, HETrack can intercept the real-time video stream into frames, and employ the trained model to generate the heatmap of current frame for gaze estimation. We implement HETrack based on a Commercial-Off-The-Shelf (COTS) camera and conduct extensive experiments to evaluate its performance. Our HETrack system only requires once calibration; whereas, the state-of-the-art work proposed by Google requires 3 similar to 5 times calibration on average. Unlike previous approaches that transmit raw image data to a central server, in our HETrack system, only parameters are transmitted, thereby well protecting the user's privacy. Experimental results demonstrate that the average distance error of estimated gaze point is 3cm, which is compatible with the state-of-the-art methods.