Task weighting based on particle filter in deep multi-task learning with a view to uncertainty and performance

Recently multi-task learning (MTL) has been widely used in different applications to build more robust models by sharing knowledge across several related tasks. However, one challenge that arises is the vari-ability in the learning pace of different tasks causing the inefficiency of naively training all tasks. There-fore, it is of great importance to consider some coefficients to balance tasks in the process of learning, but, due to the large search space and the significance of setting them properly, conventional search methods such as grid or random search are no longer effective. In this paper, we propose a learning mechanism for these coefficients based on the high efficiency of the particle filter (PF) algorithm to deal with nonlinear search problems. PF considers each state of the tasks' coefficients as a particle and recur-sively converges coefficients to an optimum point. While in most previous works coefficients were evalu-ated to only increase performance, to address the recent concerns related to applying AI in real-world ap-plications, we also incorporate uncertainty alongside our method to prevent learning coefficients leading to unstable outcomes. This mechanism is independent of the models main learning process and can be easily added to every learning system without changing its training algorithm. Extensive experiments on real-world data sets demonstrate the superiority of the proposed method over the state-of-the-art meth-ods on both performance and uncertainty. We also proved the acceptable performance of the method using Cramer Rao lower bound theory.(c) 2023 Elsevier Ltd. All rights reserved.