A modified stochastic quasi-Newton algorithm for summing functions problem in machine learning

In this paper, a new stochastic quasi-Newton method (SQN) is proposed which has a different approximation of the Hessian inverse matrix H-k. The modified quasi-Newton Broyden-Fletcher-Goldfarb-Shanno (BFGS) formula which has a better approximation to Hessian matrix has not only the gradient variation but also the function value. Because of the special nature of the sum function, the mini-batch setting is built in the algorithm, and less compution cost can be guaranteed. The number of iterations reduce to at most O(epsilon (-1/1-beta)). The convergence analysis is established in this paper. The numerical experiments show that this algorithm is competitive to other algorithms.