Multimodal Learning for Integrated Sensing and Communication Networks

Integrated sensing and communication (ISAC) is a promising technique for beyond 5G networks. In ISAC networks, the sensed environmental data may be multimodal data, which may result in high computation and communication latency due to the large size of data modalities and limited computation capability of mobile devices. To solve the problem, in this paper, we propose multimodal learning in ISAC networks. Simulation results show that the proposed multimodal learning design significantly outperforms several benchmarks without considering multimodal data sensing and communication.