SEPAKE: a structure-enhanced and position-aware knowledge embedding framework for knowledge graph completion

Knowledge Graphs (KGs) provide supportively structured knowledge and have been applied to various downstream applications. Given a large amount of incomplete knowledge in KGs, knowledge graph completion (KGC) is proposed to reason over known facts and infer the missing links. The previous graph embedding approaches learn graph structure (i.e., triple structure/neighborhood structure) but cannot handle unseen entities, which is addressed by textual encoding approaches that utilize the textual knowledge of graph elements (i.e., entities/relations). However, the previous textual encoding approaches only resort to triples and thus cannot exploit the knowledge of neighbors, which provides abundant evidence to facilitate prediction. Moreover, they are insensitive to changes in the position of elements in triples when performing text modeling, and thus cannot effectively distinguish triples with the same elements but completely different semantics, which is detrimental to the final result. To address the above challenges, we propose a novel Structure-Enhanced and Position-Aware Knowledge Embedding (SEPAKE) framework. Specifically, masked elements reconstruction is devised to predict missing elements by reasoning over the contexts of subgraphs. As such, we incorporate the graph structure while maintaining the feature that textual information can be encoded. Meanwhile, position-aware learning is conducted to capture the semantic knowledge implied by the relative positions of elements in textualization. In addition, we employ task-specific adapters to store knowledge in a unified way to facilitate the storage and transfer of knowledge. Extensive experiments demonstrate the effectiveness of our framework, and we achieve state-of-the-art performance on standard datasets compared with textual encoding approaches. Besides, our proposed framework can efficiently improve the previous approaches by optionally pluggable adapters, further verifying the advancement and applicability of our work.