An On-device Robust Sound Recognition System for Real-time Context Awareness of Robots

This paper suggests an on-device robust sound recognition system for robots in real-time. The proposed system is designed to enable the robot to detect a variety of sound events in a variety of locations, including noisy and reverberant sound environments. To use suggested system on target robots, two VGGish models are trained on sever-side and the pre-trained models infer using an audio topic from a on-device real-time buffer handling system. The buffer handling system and the training system of deep learning model are designed to get almost silmilar input audio stream with each normalization system. To get robust performance in various environments, we use log-mel feature for general environments and per-chennal energy normalization for noisy and reverberant environments. Each feature is switched and used in real time on the robot depending on the sound environment mode. Several experimental results demonstrate the robust performance of the proposed real-time robust sound recognition system on a target robot.