Learning Personalized Image Aesthetics From Subjective and Objective Attributes

被引:27
作者
Zhu, Hancheng [1 ]
Zhou, Yong [1 ]
Li, Leida [2 ]
Li, Yaqian [3 ]
Guo, Yandong [3 ]
机构
[1] China Univ Min & Technol, Sch Comp Sci & Technol, Xuzhou 221116, Peoples R China
[2] Xidian Univ, Sch Artificial Intelligence, Xian 710071, Peoples R China
[3] OPPO Res Inst, Shanghai 200032, Peoples R China
基金
中国国家自然科学基金;
关键词
Personalized image aesthetics assessment; aesthetic attributes; personality traits; aesthetic prior model; aesthetic preferences;
D O I
10.1109/TMM.2021.3123468
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Due to the widespread popularity of social media, researchers have developed a strong interest in learning the personalized image aesthetics of online users. Personalized image aesthetics assessment (PIAA) aims to study the aesthetic preferences of individual users for images, which should be affected by the properties of both users and images. Existing PIAA approaches usually use the generic aesthetics learned from images as a prior model and adapt it to PIAA models through a small number of data annotated by individual users. However, the prior model merely learns the objective attributes of images, which is agnostic to the subjective attributes of users, complicating efficient learning of the personalized image aesthetics of individual users. Therefore, we propose a personalized image aesthetics assessment method that integrates the subjective attributes of users and objective attributes of images simultaneously. To characterize these two attributes jointly, an attribute extraction module is introduced to learn users' personality traits and image aesthetic attributes. Then, an aesthetic prior model is built from numerous individual users' annotated data, which leverages the personality traits of users and the aesthetic attributes of rated images as prior knowledge to model both the image aesthetic distribution and users' residual scores relative to generic aesthetics simultaneously. Finally, a PIAA model is obtained by fine-tuning the aesthetic prior model with an individual user's annotated data. Experiments demonstrate that the proposed method is superior to existing PIAA methods in learning individual users' personalized image aesthetics.
引用
收藏
页码:179 / 190
页数:12
相关论文
共 52 条
[1]   Multitask learning [J].
Caruana, R .
MACHINE LEARNING, 1997, 28 (01) :41-75
[2]  
Cui C., IEEE T MULTIMEDIA, V21, P1209
[3]   Social-sensed Image Aesthetics Assessment [J].
Cui, Chaoran ;
Lin, Peiguang ;
Nie, Xiushan ;
Jian, Muwei ;
Yin, Yilong .
ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS, 2021, 16 (03)
[4]   Personalized image quality assessment with Social-Sensed aesthetic preference [J].
Cui, Chaoran ;
Yang, Wenya ;
Shi, Cheng ;
Wang, Meng ;
Nie, Xiushan ;
Yin, Yilong .
INFORMATION SCIENCES, 2020, 512 :780-794
[5]   Putting reward in art: A tentative prediction error account of visual art [J].
de Cruys, Sander Van ;
Wagemans, Johan .
I-PERCEPTION, 2011, 2 (09) :1035-1062
[6]   Personalized Image Aesthetics Assessment [J].
Deng, Xiang ;
Cui, Chaoran ;
Fang, Huidi ;
Nie, Xiushan ;
Yin, Yilong .
CIKM'17: PROCEEDINGS OF THE 2017 ACM CONFERENCE ON INFORMATION AND KNOWLEDGE MANAGEMENT, 2017, :2043-2046
[7]   Image Aesthetic Assessment An experimental survey [J].
Deng, Yubin ;
Loy, Chen Change ;
Tang, Xiaoou .
IEEE SIGNAL PROCESSING MAGAZINE, 2017, 34 (04) :80-106
[8]   Learning Subjective Attributes of Images from Auxiliary Sources [J].
Gelli, Francesco ;
Uricchio, Tiberio ;
He, Xiangnan ;
Del Bimbo, Alberto ;
Chua, Tat-Seng .
PROCEEDINGS OF THE 27TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA (MM'19), 2019, :2263-2271
[9]   'Who Likes What and, Why?' Insights into Modeling Users' Personality Based on Image 'Likes' [J].
Guntuku, Sharath Chandra ;
Zhou, Joey Tianyi ;
Roy, Sujoy ;
Lin, Weisi ;
Tsang, Ivor W. .
IEEE TRANSACTIONS ON AFFECTIVE COMPUTING, 2018, 9 (01) :130-143
[10]   Automatic Image Cropping for Visual Aesthetic Enhancement Using Deep Neural Networks and Cascaded Regression [J].
Guo, Guanjun ;
Wang, Hanzi ;
Shen, Chunhua ;
Yan, Yan ;
Liao, Hong-Yuan Mark .
IEEE TRANSACTIONS ON MULTIMEDIA, 2018, 20 (08) :2073-2085