An environmental perception and navigational assistance system for visually impaired persons based on semantic stixels and sound interaction

Assistive technologies aim at enhancing personal mobility of individuals with disabilities to improve their independence and access to social life. For the visually impaired, perception during navigation comprises a major ingredient of independent living. With the development of computer vision, it is possible to meet the richer needs of visually impaired people. However, research on navigation assistance for the visually impaired is still relatively unexplored when compared with the active progress in autonomous driving which is already in full swing. In respond to this issue, we aim to leverage the study of the Stixel-World for automotive systems and transfer it to develop assistive technology for visually impaired people. The impressive research results of deep learning also suppose benefits for vision-based technology. Precisely, semantic segmentation is a task that enables identification of different objects uniformly. Inspired by these observations, we design a set of wearable visual aids, while the core algorithm is based on the stixel representations for three-dimensional world combined with pixel-wise semantic segmentation. Predetermined conditions for stixels in automotive research are optimized to fit the needs of navigation assistance in our algorithm, along with the incorporation of traversability-related semantic information. We also propose a sound mapping scheme, so that the environmental awareness about geographic and semantic information are conveyed to the visually impaired through acoustic feedback.