A Sparse L2-Regularized Support Vector Machines for Large-Scale Natural Language Learning

Linear support vector machines (SVMs) have become one of the most prominent classification algorithms for many natural language learning problems such as sequential labeling tasks. Even though the L-2-regularized SVMs yields slightly more superior accuracy than L-1-SVM, it produces too much near but non zero feature weights. In this paper, we present a cutting-weight algorithm to guide the optimization process of L-2-SVM into sparse solution. To verify the proposed method, we conduct the experiments with three well-known sequential labeling tasks and one dependency parsing task. The result shows that our method achieved at least 400% feature parameter reduction rates in comparison to the original L-2-SVM, with almost no change in accuracy and training times. In terms of run time efficiency, our method is faster than the original L-2-regularized SVMs at least 20% in all tasks.