Deep Reinforcement Learning in Continuous Action Spaces for Pair Trading: A Comparative Study of A2 C and PPO

Reinforcement learning (RL) is revolutionizing financial trading strategies by offering adaptive solutions for complex market behaviors. This paper examines the application of two RL algorithms-advantage actor-critic (A2 C) and proximal policy optimization (PPO)-in implementing a pair trading strategy, a market-neutral approach that leverages the price dynamics between correlated assets. Using historical price data from S&P 500 assets, we train and test these models, comparing their performance with the industry-standard benchmark based on mean deviation. In this way, results demonstrate that RL-based strategies achieve positive investment metrics for most pairs tested. Specifically, the A2 C algorithm achieved an average Sharpe ratio of 0.42 and a Sortino ratio of 0.63 for the AAP-SO pair, while the PPO algorithm achieved a Sharpe ratio of 0.41 and a Sortino ratio of 0.62 for the same pair. However, the benchmark model outperforms the RL models in several cases, suggesting opportunities for further refinement. This research underscores both the viability and limitations of RL in optimizing pair trading strategies within real-world constraints. © The Author(s), under exclusive licence to Springer Nature Singapore Pte Ltd. 2025.