A convergence rate estimate for the SVM decomposition method

The training of Support Vector Machines using the decomposition method has one drawback; namely the selection of working sets such that convergence is as fast as possible. It has been shown by Lin that the rate is linear in the worse case under the assumption that all bounded Support Vectors have been determined. The analysis was done based on the change in the objective function and under a SVMlight selection rule. However, the rate estimate given is independent of time and hence gives little indication as to how the linear convergence speed varies during the iteration. In this initial analysis, we provide a treatment of the convergence from a gradient contraction perspective. We propose a necessary and sufficient condition which when satisfied provides strict linear convergence of the algorithm. The condition can also be interpreted as a basic requirement for a sequence of working sets in order to achieve such a convergence rate. Based on this condition, a time dependant rate estimate is then further derived. This estimate is shown to monotonically approach unity from below.