ARL: analogical reinforcement learning for knowledge graph reasoning

Reinforcement Learning (RL) knowledge graph reasoning aims to predict complete triplets by learning existing relationship paths. This greatly improves the efficiency of prediction because the RL-based methods do not traverse all entities and relations like representation reasoning. Meanwhile, this kind of method increases the interpretability of reasoning. However, due to the necessity of normalizing the entity outdegree matrices for neural network computations in each step of the retrieval process in reinforcement learning, entities with an excessively high number of outdegrees compel the RL-based model to restrict the retrieval space of each path. Consequently, this limitation leads to the omission of some correct answers. Moreover, for some isolated tail entities with sparse connections, this path-based reasoning will lose these island nodes. To solve both problems, we propose an analogy-based reinforcement learning model named Analogical Reinforcement Learning network (ARL). This model features a novel analogy reinforcement learning architecture, dynamic graph attention networks, and our proprietary AODS algorithm. It injects entity analogy information into the model's reasoning process and employs virtual link generation, which not only enhances the probability of paths getting rewards, but also increases the breadth of path connection and brings more possibilities for island nodes. In the meantime, we analyze and compare various analogy methods in detail. Experimental results show that ARL outperforms existing multi-hop methods on several datasets.