Multimodal Learning-Based Interval Type-2 Fuzzy Neural Network

Interval type-2 fuzzy neural network (IT2FNN) has extensive applications for modeling nonlinear systems with multidimensional structured data. However, the traditional IT2FNN based on the structured topology struggles to identify nonlinear systems using semistructured and unstructured data. To tackle this issue, a multimodal learning-based IT2FNN (ML-IT2FNN) is developed for joint learning of the multimodal data. First, an encoding layer with a multimodal perception strategy is designed to identify the multimodal information. The parameterized modalities are utilized to map the features of the semistructured and unstructured data into the structured spaces. Second, a multimodal representation mechanism is introduced to extract the features of multiple modalities from the structured spaces. In this mechanism, type-2 fuzzy sets with soft boundaries are used to intricate coupling relationships among modalities by adapting to the nuances of multimodal data. Third, a constrained hybrid learning algorithm, combining parallel and sequential updating frameworks, is presented to optimize the parameters of ML-IT2FNN. The type-2 fuzzy parameters and the coupling parameters with constraints are updated adaptively to facilitate the intramodal identification performance and cross-modal interaction performance. Finally, a series of examples in nonlinear systems are introduced to verify ML-IT2FNN. Empirical results demonstrate that ML-IT2FNN surpasses the cutting-edge approaches with accuracy.