Deep Interaction Machine: A Simple but Effective Model for High-order Feature Interactions

Click-Through Rate (CTR) prediction is a crucial task for various online applications, such as recommendation and online advertising. The task of CTR prediction is to predict the probability of users' clicking behaviors, with high-dimensional input features. To avoid heavy handcrafted feature engineering, the core topic of CTR prediction is the automatic interactions of the input features. Factorization Machine (FM) is an effective approach for modeling second-order feature interactions. Recently, FM has been extended for modeling higher-order feature interactions, such as xDeepFM and Higher-Order Factorization Machine (HOFM). However, these approaches are with either high complexity or iterative computation consuming much time and space. To overcome above problems, we express arbitrary-order FM in the form of power sums according to Newton's identities. Accordingly, we propose a novel Interaction Machine (IM) model. IM is an efficient and exact implementation of high-order FM, whose time complexity linearly grows with the order of interactions and the number of feature fields. Via IM, we can conduct arbitrary-order feature interactions in a very simple way. Moreover, we perform IM together with deep neural networks, and the resulted DeepIM model is more efficient than xDeepFM with comparable or even better performance. We conduct experiments on two real-world datasets, in which effectiveness and efficiency of both IM and DeepIM are strongly verified.