A Multi-Step Reinforcement Learning Algorithm

Reinforcement learning (RL) is a state or action value based machine learning method which approximately solves large-scale Markov Decision Process (MDP) or Semi-Markov Decision Process (SMDP). A multi-step RL algorithm called Sarsa(lambda,k) is proposed, which is a compromised variation of Sarsa and Sarsa(lambda). It is equivalent to Sarsa if k is 1 and is equivalent to Sarsa(lambda) if k is infinite. Sarsa(lambda,k) adjust its performance by setting k value. Two forms of Sarsa(lambda,k), forward view Sarsa(lambda,k) and backward view Sarsa(lambda,k), are constructed and proved equivalent in off-line updating.