Transformer-Based Approach for Solving Mathematical Problems Using Automatic Speech Recognition

In this paper, we introduce Vox Calculi, a system designed to solve mathematical problems using voice transcriptions. By leveraging state-of-the-art pretrained Automatic Speech Recognition (ASR) models, we accurately transcribe users' voice recordings. Additionally, we develop a specialized mathematical parser that converts natural language mathematical expressions into symbolic representations and numerical values while preserving the remainder of the transcription. We utilize Transformer and TP-Transformer models, trained on DeepMind's Mathematics dataset, to generate an appropriate mathematical answer from the provided input sequence. An in-depth evaluation of both the ASR and Transformer models demonstrates highly satisfactory results. Furthermore, we propose potential future improvements to enhance the system's performance.