A Convex Model for Support Vector Distance Metric Learning

Distance metric learning (DML) aims to learn a distance metric to process the data distribution. However, most of the existing methods are kNN DML methods and employ the kNN model to classify the test instances. The drawback of kNN DML is that all training instances need to be accessed and stored to classify the test instances, and the classification performance is influenced by the setting of the nearest neighbor number k. To solve these problems, there are several DML methods that employ the SVM model to classify the test instances. However, all of them are nonconvex and the convex support vector DML method has not been explicitly proposed. In this article, we propose a convex model for support vector DML (CSV-DML), which is capable of replacing the kNN model of DML with the SVM model. To make CSV-DML can use the most kernel functions of the existing SVM methods, a nonlinear mapping is used to map the original instances into a feature space. Since the explicit form of nonlinear mapped instances is unknown, the original instances are further transformed into the kernel form, which can be calculated explicitly. CSV-DML is constructed to work directly on the kernel-transformed instances. Specifically, we learn a specific Mahalanobis distance metric from the kernel-transformed training instances and train a DML-based separating hyperplane based on it. An iterated approach is formulated to optimize CSV-DML, which is based on generalized block coordinate descent and can converge to the global optimum. In CSV-DML, since the dimension of kernel-transformed instances is only related to the number of original training instances, we develop a novel parameter reduction scheme for reducing the feature dimension. Extensive experiments show that the proposed CSV-DML method outperforms the previous methods.