Towards Deeper Understanding of Variational Auto-encoders for Binary Collaborative Filtering

Recommendation systems are an integral component of machine learning, wherein collaborative filtering (CF) is among the most prominent algorithms employed. Recently, variational auto-encoders (VAEs) with multinomial likelihood and weighted Kullback-Leibler (KL) regularization (referred to as Mult-VAE) provide state-of-the-art performance for collaborative filtering of binary data. To gain deeper insight into the objective function of Mult-VAE, we build a connection between the reconstruction term of Mult-VAE objective and the objective function of the probabilistic n-Choose-k model for ranking prediction. In particular, we theoretically demonstrate that the negative reconstruction error of Mult-VAE is a lower bound to the log-likelihood of the binary n-Choose-k model. Hence, Mult-VAE can be interpreted as an approximate proxy to the n-Choose-k model. We also empirically show the essential role of this reconstruction term of evidence lower bound in the context of collaborative filtering on multiple real-world datasets. Finally, inspired by the role of the weighted KL term in maximizing mutual information between observed ratings and latent variables, we propose a semi-implicit VAE framework with superior performance in terms of ranking metrics.