Robust Estimator based Adaptive Multi-Task Learning

Multi-Task Learning(MTL) algorithm leverages valuable information across similar tasks to improve regression accuracy. However, applications in practice often encounter scenes that cannot give an accurate relationship between the large-scale number of multiple tasks. When task clustering based method is employed in those cases, excessive connections between dissimilar tasks and meaningless dimensions would cause a poorly predictive performance, which cannot be perfectly interpreted by existing MTL methods. In this paper, a novel MTL method is proposed to make more accurate use of valuable shared information between multiple tasks, where a redescending robust estimator is utilized to adaptively unify the continuous clustering of a large-scale number of tasks and dynamically selecting valuable features of few-shot tasks. To enable a better description of relationships between multiple tasks, we formulate a multi-convex objective function that can be optimized alternatively. After analyzing the complexity and convexity of the problem, we provide a scalable solving approach which can converge to the optimum with approximately linear time complexity. Compared with state-of-the-art models, the proposed approach performs better RMSE score and time efficiency both in synthetic and realistic datasets. Meanwhile, with similar computational overhead, the experiment demonstrates that our method has better regression accuracy than clustering tasks alone or selecting valuable features individually.