Contactless human-computer interaction system based on three-dimensional holographic display and gesture recognition

Contactless interaction provides a hygienic, safe, and intelligent way for communication between human and computers. As computing systems continue to advance, there is a growing focus on input methods and keyboard interaction approaches. However, current research on virtual keyboard systems remains confined to two-dimensional interfaces. The existing contactless interaction implementations are generally limited by handheld tools, wearable devices, or projection screens. In this work, we propose a contactless three-dimensional (3D) virtual keyboard system, which combines gesture recognition and holographic display. A hand-tracking sensor is used to collect the user's gestures and fingertip positions as input. A spatial light modulator is utilized to generate the corresponding holographic 3D display images. The hand-tracking sensor and display module are controlled in a synchronous manner by a personal computer. The robust and real-time system is demonstrated through user-interactive experiments with a 3D virtual keyboard. The scalability of the proposed human-computer interaction system supports a wide range of applications, including education, communication, disease prevention, information security, and virtual reality.