Using networks to visualize, analyse and interpret multimodal communication

Multimodality is a virtually ubiquitous feature of communication. With the increasing interest in how animals, including humans, use multimodal and multicomponent signals in social interactions, there is an acute need for standardized and rigorous tools that will allow us to visualize and analyse these signals as they occur in naturalistic interactions as a complex, integrated system. Network theory is a powerful methodology for intuitively visualizing and investigating the relationships between entities. Here, we propose a new framework for analysing multimodal communication. Using a case study of natural multimodal interactions in wild plains zebras, Equus quagga, we introduce the descriptive power of network metrics by providing an objective set of metrics to (1) describe the relationships between simultaneously produced signals within and between modalities and (2) infer signal meaning and function. We embed these tools in a theoretical framework that can be used to interpret and describe both the global structure of the repertoire and the role individual signals play in shaping and modulating meaning. Next, we review an array of common questions in animal behaviour that could benefit from the use of multimodal networks to facilitate meaningful comparisons of communication across social and environmental contexts, timescales and species. Finally, we discuss extending the use of network analyses to multimodal communication through the use of directed networks and the challenges to be overcome from this application. (c) 2023 The Association for the Study of Animal Behaviour. Published by Elsevier Ltd. All rights reserved.