Audiovisual Spatial-Audio Analysis by Means of Sound Localization and Imaging: A Multimedia Healthcare Framework in Abdominal Sound Mapping

This paper presents the novel sound-field localization and visualization techniques for audiovisual spatial audio analysis, content description, and management. The method focuses on topographic analysis and mapping of gastro-intestinal motility (GIM) through multichannel recording of abdominal sounds (AS). Related research has attempted to study GIM physiology patterns and diagnose specific abnormalities or diseases. In this context, a new AS source localization method is implemented and evaluated. Sound imaging and spatiotemporalmapping utilities are deployed next by means of sound level distribution images. Novel audio description and management is introduced for the analysis automation of prolonged AS recordings, facilitating smart content browsing with audiovisual summarization and highlighting. The proposed modalities can be utilized in multimedia healthcare applications, where multimodal monitoring of GIM and other psychophysiological parameters can be combined for the study of human digestive patterns and their relation to other factors (i.e., subjects' medical history, nutrition, medication, psychological state, and others). The adopted methodology of spatial audio analysis introduces a generic framework that can be efficiently applied to various sound localization and imaging tasks, which are encountered both in bioacoustics and in contemporary multimedia involving multichannel/3D audio.