Cross-sentence Pre-trained model for Interactive QA matching

Semantic matching measures the dependencies between query and answer representations, which is an important criterion for evaluating whether the matching is successful. In fact, such matching does not examine each sentence individually, because the context information between sentences should be considered equally important to the syntactic context inside a sentence. Considering the above, we propose a novel QA matching model, built upon a cross-sentence context-aware architecture. Specifically, an interactive attention mechanism with a pre-trained language model is presented to automatically select salient positional answer representations that contribute more significantly to the answer relevance of a given question. In addition to the context information captured at each word position, we incorporate a quantity of context jump dependencies to leverage the attention weight formulation. This can capture the amount of useful information brought by the next word, is computed by modeling the joint probability between two adjacent word states. The proposed method is compared with multiple state-of-the-art methods using the TREC library, WikiQA, and the Yahoo! community question datasets. Experimental results show that the proposed method outperforms satisfactorily the competing ones.