Transfer learning in constructive induction with Genetic Programming

Transfer learning (TL) is the process by which some aspects of a machine learning model generated on a source task is transferred to a target task, to simplify the learning required to solve the target. TL in Genetic Programming (GP) has not received much attention, since it is normally assumed that an evolved symbolic expression is specifically tailored to a problem's data and thus cannot be used in other problems. The goal of this work is to present a broad and diverse study of TL in GP, considering a varied set of source and target tasks, and dealing with questions that have received little, or no attention, in previous GP literature. In particular, this work studies the performance of transferred solutions when the source and target tasks are from different domains, and when they do not share a similar input feature space. Additionally, the relationship between the success and failure of transferred solutions is studied, considering different source and target tasks. Finally, the predictability of TL performance is analyzed for the first time in GP literature. GP-based constructive induction of features is used to carry out the study, a wrapper-based approach where GP is used to construct feature transformations and an additional learning algorithm is used to fit the final model. The experimental work presents several notable results and contributions. First, TL is capable of generating solutions that outperform, in many cases, baseline methods in classification and regression tasks. Second, it is shown that some problems are good source problems while others are good targets in a TL system. Third, the transferability of solutions is not necessarily symmetric between two problems. Finally, results show that it is possible to predict the success of TL in some cases, particularly in classification tasks.