Survey of Multi-Task Learning

With the development of artificial intelligence technology such as image processing and speech recognition, many learning methods, especially those using deep learning frameworks, have achieved excellent performance and greatly improved accuracy and speed, but the problems are also obvious, if these learning methods want to achieve a stable learning effect, they often need to use a large number of labeled data to train adequately. Otherwise, there will be an under-fitting situation which will lead to the decline of learning performance. Therefore, with the increase of task complexity and data scale, higher requirements are put forward for the quantity and quality of manual labeling data, resulting in the increase of labeling cost and difficulty. At the same time, the independent learning of single task often ignores the experience information from other tasks, which leads to redundant training and waste of learning resources, and also limits the improvement of its performance. In order to alleviate these problems, the multi-task learning method, which belongs to the category of transfer learning, has gradually attracted the attention of researchers. Unlike single-task learning, which only uses sample information of a single task, multi-task learning assumes that there is a certain similarity between the data distribution of different tasks. On this basis, the relationship between tasks is established through joint training and optimization. This training mode fully promotes information exchange between tasks and achieves the goal of mutual learning. Especially under the condition that the sample size of each task is limited, each task can get some inspiration from other tasks. With the help of information transfer in the learning process, the data of other tasks can be indirectly utilized. Thus, the dependence on a large number of labeled data is alleviated, and the goal of improving the performance of task learning is also achieved. Under this background, this paper first introduces the concept of related tasks, and describes their characteristics one by one after classifying the types of related tasks according to their functions. Then, according to the data processing mode and task relationship modeling process, the current mainstream algorithms are divided into two categories: structured multi-task learning algorithm and deep multi-task learning algorithm. The structured multi-task learning algorithm adopts linear model, which can directly assume the structure of the data and express the task relationship with the original annotation features. At the same time, it can be subdivided into two different structures based on task level and feature level according to the different learning objects. Each structure has two implementation means: discriminant method and generative method. Different from the modeling process of structured multi-task learning algorithm, deep multi-task learning algorithm uses the deep information abstracted by multi-layer features to describe the task relationship, and achieves the goal of information sharing by processing the parameters in the specific network layer. Then, taking two kinds of algorithms as the main line, this paper analyzed the structural assumptions, implementation approaches, advantages and disadvantages of different modeling methods and the relationship between them in detail. Finally, this paper summarizes the criteria for identifying the similarity and compactness between tasks, and the effectiveness and intrinsic causes of multi-task mechanism are also analyzed, then the characteristics of multi-task information migration are expounded from the perspectives of inductive bias and dynamic solution. © 2020, Science Press. All right reserved.