A Parallelized Combined Direction Simultaneous Perturbation Stochastic Approximation Algorithm

The simultaneous perturbation stochastic approximation(SPSA) belongs to the class of iterative gradient-free algorithm. However, because of its slow convergence rate, the experimental effect is not ideal for large-scale problems. In order to accelerate the SPSA algorithm, this paper proposes a parallelized combined direction SPSA algorithm. Gradient directions among the master and slaves are combined to be a new iteration direction in every iteration to improve algorithm performance. Numerical experiments for typical nonlinear optimization problems demonstrate that the present algorithm outperforms the SPSA algorithm in the fewer step numbers.