EarGest: Hand Gesture Recognition with Earables

Earables have been increasingly gaining attention from consumers and manufacturers alike due to their small footprint, ease of use, and the added accessibility they bring. However, the limited interface of these devices, usually being a single button or force-sensor, inhibits their potential. Earables can provide a much richer experience by extending the ways in which users can interact with them. In this paper, we present EarGest, a novel earable-based hand gesture recognition system that does not require calibration or training, and works with commercially available BLE-enabled earphones and devices. Our proposed system is unique in its ability to leverage Bluetooth to detect hand motion near the ear to recognize gestures. By harnessing information from BLE connections between the wireless earphones and a host device, we accurately detect and classify different hand gestures performed by users, while also determining discrete levels of hand speed. EarGest operates without interfering with the regular functionality of the earphones and introduces minimal energy overhead on the host device. We implement a prototype of the system using eSense, a multi-sensory earable platform, and evaluate it in different scenarios and settings. Results show that our system can detect and classify seven near-ear hand gestures with an accuracy up to 98.5%, as well as identify hand motion speed with 96% accuracy.