Learning classifier systems with memory condition to solve non-Markov problems

In the family of learning classifier systems, the classifier system XCS has been successfully used for many applications. However, the standard XCS has no memory mechanism and can only learn optimal policy in Markov environments, but fails in non-Markov ones. In this work, we aim to develop a new classifier system based on XCS to tackle this problem. It adds a memory list with numbered slots to XCS to record input sensation history, and extends only a small number of classifiers with memory conditions. The classifier’s memory condition, as a foothold to disambiguate non-Markov states, is used to sense a specified element in the memory list, which makes our system can “jump over” irrelevant or confusing states to get decisive prior information that may be far back in time. Besides, a detection method is employed to recognize non-Markov states in environments, to avoid these states controlling over classifiers’ memory conditions. Furthermore, four sets of different complex maze environments have been tested by the proposed method. Experimental results show that our system can overcome the overhead problem often encountered in history-window approaches, and is an effective technique to solve non-Markov environments.