Pseudo-haptic Feedback Design for Virtual Activities in Human Computer Interface

Digital world such as Metaverse requires a fully multimodal integrated cognitive interface to provide realistic and immersive interaction that boosts user engagement. In this paper, cross-modality (visual and audio, and/or haptic) driven pseudo haptic interface has been investigated in AR environment. In-house haptic platform named "Haptic Signal Processing Platform" (HSPP) has been implemented to detect any modalities in real-time. For user test, five primitive virtual activities including click, scroll, object manipulation, zoom-in and zoom-out are considered using visual& audio and multimodal (visual& audio, and haptic) feedbacks. Cross-modality feedback using HSPP has been psychophysically evaluated in terms of three categories; intuitiveness, attractiveness and immersiveness. To evaluate above three qualitative categories, NASA-TLX has been implemented. This evaluation method is recommended to measure the performance of a particular individual in a task; mental, physical, temporal, performance, effort and frustration. The results clearly show that the cross-modality based pseudo-haptic with HSPP can produce higher performance in terms of intuitiveness, attractiveness and immersiveness.