Few-shot intent detection with mutual information and contrastive learning

Few-shot intent detection is a challenging task. Most existing methods only focus on acquisition of generalization knowledge in known classes, or on the adaptation situation of meta-learning tasks. However, these methods lack both the learning of beneficial information for intent detection and the learning of discriminative representation of intent keywords. Solving these problems can effectively improve the model's ability to recognize text intent and thus effectively execute user commands. Therefore, in this paper, we propose a few-shot intent detection framework based on Mutual Information and Contrastive Learning (MICL). It improves the robustness of the model learning of intent keywords through mutual information maximization, thus improving the accuracy of intent detection. Additionally, maximization of mutual information enhances the intent representation consistency on two text views,1which helps to promote the efficiency of sample contrastive learning. Specifically, we use a neural network to approximately estimate the joint probability distribution and marginal probability distribution of two text views and maximize the mutual information through back-propagation. In order to enable the model to focus on intent keywords in text, we introduce a sample-level contrastive learning. It helps the model to learn discriminative representations of intent keywords by comparing the similarity of text from two text views. In addition, to prevent overfitting of the model on known classes, we introduce a class-level contrastive learning. We regard it as regularization of known classes, aiming to improve the model generalization performance on unknown classes. Our method achieves state-of-the-art performance on four public datasets.2