3MN: Three Meta Networks for Multi-Scenario and Multi-Task Learning in Online Advertising Recommender Systems

Recommender systems are widely applied on web. For example, online advertising systems rely on recommender systems to accurately estimate the value of display opportunities, which is critical to maximize the profits of advertisers. To reduce computational resource consumption, the core tactic of Multi-Scenario Multi-Task Learning (MSMTL) is to devise a single recommder system that is adapted to all contexts instead of implementing multiple scenario-oriented or task-oriented recommender systems. However, MSMTL is challenging because there are complicated task-task, scenario-scenario, and task-scenario interrelations; the characteristic of different tasks in different scenarios also largely varies; and samples of each context are often unevenly distributed. Previous MSMTL solutions focus on applying scenario knowledge to improve the performance of multi-task learning, while neglecting the complicated interrelations among tasks and scenarios. Moreover, samples derived from different scenarios are transferred into the latent embedding with the same dimension. This static embedding strategy impedes the practicality of model expressiveness, since the scenarios with sufficient samples are underrepresented and those with insufficient samples are over-represented. In this paper, we propose a novel three meta networks-based solution (3MN) to MSMTL that addresses all the limitations discussed above. Specifically, we innovatively bind the meta network with scenario-related input in bottom embedding layer, so that the embedding layer is capable of learning the scenario-related knowledge explicitly. To counteract the imbalanced scenario-related data distributions, our flexible embedding layer adaptively learns the representation of samples. This innovative embedding layer is also able to boost other solutions as a plug-in. Moreover, to fully capture the interrelations among scenarios and tasks, we enforce the task and scenario information into the other two meta networks, and transfer the resulted meta-knowledge into the top components (i.e., backbone network and classifier) of the recommender system, respectively. These three meta networks contribute to the superiority of our 3MN solution over state-of-the-art MSMTL solutions, which is demonstrated by extensive offline experiments. 3MN has been successfully deployed in our industrial online advertising system.