Values of User Exploration in Recommender Systems

被引:25
作者
Chen, Minmin [1 ]
Wang, Yuyan [1 ]
Xu, Can [1 ]
Le, Ya [1 ]
Sharma, Mohit [1 ]
Richardson, Lee [1 ]
Wu, Su-Lin [1 ]
Chi, Ed [1 ]
机构
[1] Google, Mountain View, CA 94043 USA
来源
15TH ACM CONFERENCE ON RECOMMENDER SYSTEMS (RECSYS 2021) | 2021年
关键词
reinforcement learning; exploration; serendipity; recommender systems; DIVERSITY;
D O I
10.1145/3460231.3474236
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Reinforcement Learning (RL) has been sought after to bring next-generation recommender systems to further improve user experience on recommendation platforms. While the exploration-exploitation tradeoff is the foundation of RL research, the value of exploration in (RL-based) recommender systems is less well understood. Exploration, commonly seen as a tool to reduce model uncertainty in regions of sparse user interaction/feedback, is believed to cost user experience in the short term, while the indirect benefit of better model quality arrives at a later time. We focus on another aspect of exploration, which we refer to as user exploration to help discover new user interests, and argue it can improve user experience even in the more imminent term. We examine the role of user exploration in changing different facets of recommendation quality that more directly impact user experience. To do so, we introduce a series of methods inspired by exploration research in RL to increase user exploration in an RL-based recommender system, and study their effect on the end recommendation quality, more specifically, on accuracy, diversity, novelty and serendipity. We propose a set of metrics to measure (RL based) recommender systems in these four aspects and evaluate the impact of exploration-induced methods against these metrics. In addition to the offline measurements, we conduct live experiments on an industrial recommendation platform serving billions of users to showcase the benefit of user exploration. Moreover, we use conversion of casual users to core users as an indicator of the holistic long-term user experience and study the values of user exploration in helping platforms convert users. Through offline analyses and live experiments, we study the correlation between these four facets of recommendation quality and long term user experience, and connect serendipity to improved long term user experience.
引用
收藏
页码:85 / 95
页数:11
相关论文
共 72 条
[1]  
Achiam J., 2017, CoRR
[2]  
Adomavicius G., 2011, P 1 INT WORKSHOP NOV, P3
[3]  
Agrawal S., 2012, C LEARN THEOR, P31
[4]   Algorithmic Effects on the Diversity of Consumption on Spotify [J].
Anderson, Ashton ;
Maystre, Lucas ;
Mehrotra, Rishabh ;
Anderson, Ian ;
Lalmas, Mounia .
WEB CONFERENCE 2020: PROCEEDINGS OF THE WORLD WIDE WEB CONFERENCE (WWW 2020), 2020, :2155-2165
[5]  
[Anonymous], 2004, P 21 INT C MACHINE L
[6]  
[Anonymous], 2011, Advances in neural information processing systems
[7]  
Bellemare MG, 2016, ADV NEUR IN, V29
[8]   Latent Dirichlet allocation [J].
Blei, DM ;
Ng, AY ;
Jordan, MI .
JOURNAL OF MACHINE LEARNING RESEARCH, 2003, 3 (4-5) :993-1022
[9]  
Castells Pablo, 2011, NOVELTY DIVERSITY ME
[10]  
Celma O, 2008, RECSYS'08: PROCEEDINGS OF THE 2008 ACM CONFERENCE ON RECOMMENDER SYSTEMS, P179