Varying Infimum Gradient Descent Algorithm for Agent-Sever Systems Using Different Order Iterative Preconditioning Methods

In the traditional gradient descent (T-GD) algorithm, the convergence rate is strongly depend on the condition number of the information matrix: a larger condition number leads to a poor optimal convergence factor infimum mu(op), which sets a convergence rate ceiling. That is, once the information matrix is fixed, the convergence factor of the T-GD algorithm reaches at most the infimum mu(op). This article studies a varying infimum gradient descent algorithm, which can move down the infimum by using different order iterative preconditioning methods, as follows: first, for infinite iterative algorithm, the infimum becomes smaller and smaller with the increased iteration numbers; second, for finite iterative algorithm, the infimum is equal to zero, and the parameter estimates can be obtained in only one iteration; third, construct an adaptive interval between zero and mu(op), which can establish a link between the least squares and T-GD algorithms. Based on the varying infimum gradient descent algorithm, researchers can adaptively choose preconditioning matrices for different kinds of models on a case by case basis. The convergence analysis and simulation examples show effectiveness of the proposed algorithms.