Recall Mechanism and Multi-Head Attention for Numerical Reasoning

Numerical reasoning is a challenging question-answering task in artificial intelligence (AI) that requires both reading comprehension and numerical computation capabilities. Although recent approaches have made significant progress in reasoning, two critical issues remain: (1) information tends to be gradually lost as the network deepens due to the complexity of deep learning models, and (2) the performance of multi-step reasoning is suboptimal, leaving room for improvement in accuracy. To address these issues, we propose a model with a recall mechanism and multi-head attention for numerical reasoning (RMMANR). The recall mechanism prevents the embedding information of questions and passages from being forgotten as the model deepens, while the multi-head attention mechanism analyzes possible solutions for numerical reasoning. RMMANR consists of two main components: an encoder and a decoder. The encoder leverages RoBERTa to encode the question and passage into contextual embeddings. The decoder, which consists of four modules (RM Module, Selector, MA Module, and Program Solver), generates inference steps and answers based on contextual information from the encoder. We implement our model using PyTorch and evaluate it on the FINQA dataset, a benchmark for numerical reasoning in the financial domain. Experimental results demonstrate that RMMANR outperforms several baseline models, achieving superior accuracy.