Adaptive learning by a target-tracking system

Purpose - The purpose of this paper is to report upon research into developing a biologically inspired target-tracking system (TTS) capable of acquiring quality images of a known target type for a robotic inspection application. Design/methodology/approach - The approach used in the design of the TTS hearkens back to the work on adaptive learning by Oliver Selfridge and Chris J.C.H. Watkins and the work on the classification of objects by Zdzislaw Pawlak during the 1980s in an approximation space-based form of feedback during learning. Also, during the 1980s, it was Ewa Orlowska who called attention to the importance of approximation spaces as a formal counterpart of perception. This insight by Orlowska has been important in working toward a new form of adaptive learning useful in controlling the behaviour of machines to accomplish system goals. The adaptive learning algorithms presented in this paper are strictly temporal difference methods, including Q-learning, sarsa, and the actor-critic method. Learning itself is considered episodic. During each episode, the equivalent of a Tinbergen-like ethogram is constructed. Such an ethogram provides a basis for the construction of an approximation space at the end of each episode. The combination of episodic ethograms and approximation spaces provides an extremely effective means of feedback useful in guiding learning during the lifetime of a robotic system such as the TTS reported in this paper. Findings - It was discovered that even though the adaptive learning methods were computationally more expensive than the classical algorithm implementations, they proved to be more effective in a number of cases, especially in noisy environments. Originality/value - The novelty associated with this work is the introduction of an approach to adaptive adaptive learning carried out within the framework of ethology-based approximation spaces to provide performance feedback during the learning process.