Understanding More Knowledge Makes the Transformer Perform Better in Document-level Relation Extraction

Relation extraction plays a vital role in knowledge graph construction. In contrast with the traditional relation extraction on a single sentence, extracting relations from multiple sentences as a whole will harvest more valuable and richer knowledge. Recently, the Transformer-based pre-trained language models (TPLMs) are widely adopted to tackle document-level relation extraction (DocRE). Graph-based methods, aiming to acquire knowledge between entities to form entity-level relation graphs, have facilitated the rapid development of DocRE by infusing their proposed models with the knowledge. However, beyond entity-level knowledge, we discover many other kinds of knowledge that can aid humans to extract relations. It remains unclear whether and in which way they can be adopted to improve the performance of the Transformer, which affects the maximum performance gain of Transformer-based methods. In this paper, we propose a novel weighted multi-channel Transformer (WMCT) to infuse unlimited kinds of knowledge into the vanilla Transformer. Based on WMCT, we also explore five kinds of knowledge to enhance both its reasoning ability and expressive power. Our extensive experimental results demonstrate that: (1) more knowledge makes the performance of the Transformer better and (2) more informative knowledge leads to more performance gain. We appeal to future Transformer-based work to consider exploring more informative knowledge to improve the performance of the Transformer.