Multimodal Recommendation Algorithm Based on Contrastive Learning and Semantic Enhancement

被引：0

作者：

Zhang, Kaihan ^{[1
]}

Feng, Chenjiao ^{[2
]}

Yao, Kaixuan ^{[3
]}

Song, Peng ^{[4
]}

Liang, Jiye ^{[3
]}

机构：

[1] School of Computer Science and Technology, North University of China, Taiyuan

[2] School of Applied Mathematics, Shanxi University of Finance and Economics, Taiyuan

[3] Key Laboratory of Computational Intelligence and Chinese Information Processing, Ministry of Education, Shanxi University, Taiyuan

[4] School of Economics and Management, Shanxi University, Taiyuan

来源：

Moshi Shibie yu Rengong Zhineng/Pattern Recognition and Artificial Intelligence | 2024年 / 37卷 / 06期

基金：

中国国家自然科学基金;

关键词：

Contrastive Learning; Graph Neural Network; Multimodal Recommendation Algorithm; Recommender System;

D O I：

10.16451/j.cnki.issn1003-6059.202406001

中图分类号：

学科分类号：

摘要：

The multimodal data of items is typically introduced into recommendation algorithms as additional auxiliary information to enrich the representation features of users and items. How to effectively integrate the interaction information with multimodal information of users and items is a key issue to the research. Existing methods are still insufficient in feature fusion and semantic association modeling. Therefore, a multimodal recommendation algorithm based on contrastive learning and semantic enhancement is proposed from the perspective of feature fusion. Firstly, the graph neural network and attention mechanism are adopted to fully integrate collaborative features and multimodal features. Next, the semantic association structures within each modality are learned under the guidance of the interaction structure in collaborative information. Meanwhile, the contrastive learning paradigm is employed to capture cross-modal representation dependencies. A reliability factor is introduced into the contrastive loss to adaptively adjust the constraint strength of the multimodal features, consequently suppressing the influence of data noise. Finally, the aforementioned tasks are jointly optimized to generate recommendation results. Experimental results on four real datasets show that the proposed algorithm yields excellent performance. © 2024 Science Press. All rights reserved.

引用

页码：479 / 490

页数：11

共 28 条

[1] ZHANG S Y, FENG F L, KUANG K, Et al., Personalized Latent Structure Learning for Recommendation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 45, 8, pp. 10285-10299, (2023)
[2] NGIAM J Q, KHOSLA A, KIM M, Et al., Multimodal Deep Learning, Proc of the 28th International Conference on Machine Learning, pp. 689-696, (2011)
[3] AITIS T, AHUJA C, MORENCY L P., Multimodal Machine Learning: A Survey and Taxonomy, IEEE Transactions on Pattern Analysis and Machine Intelligence, 41, 2, pp. 423-443, (2019)
[4] LI S Y, GUO D, LIU K, Et al., Multimodal Counterfactual Learning Network for Multimedia-Based Recommendation, Proc of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1539-1548, (2023)
[5] LIU H, WEI Y W, LIU F, Et al., Dynamic Multimodal Fusion via Meta-Learning Towards Micro-video Recommendation, ACM Transactions on Information Systems, 42, 2, (2024)
[6] HE R N, MCAULEY J J., VBPR: Visual Bayesian Personalized Ranking from Implicit Feedback, Proc of the 30th AAAI Conference on Artificial Intelligence, pp. 144-150, (2016)
[7] ZHOU Y, GUO J, SUN H, Et al., Attention-Guided Multi-step Fusion: A Hierarchical Fusion Network for Multimodal Recommendation, Proc of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1816-1820, (2023)
[8] LIU F, CHEN H L, CHENG Z Y, Et al., Disentangled Multimodal Representation Learning for Recommendation, IEEE Transactions on Multimedia, 25, pp. 7149-7159, (2023)
[9] WAGN Q F, WEI Y W, YIN J H, Et al., DualGNN: Dual Graph Neural Network for Multimedia Recommendation, IEEE Transactions on Multimedia, 25, pp. 1074-1084, (2023)
[10] HUANG Z H, LIN X L, SUN S L, Et al., Feature Augmentation based Graph Neural Recommendation Method in Session Scenarios, Chinese Journal of Computers, 45, 4, pp. 766-780, (2022)

← 1 2 3 →