Learning from pairwise constraints by Similarity Neural Networks

In this paper we present Similarity Neural Networks (SNNs), a neural network model able to learn a similarity measure for pairs of patterns, exploiting a binary supervision on their similarity/dissimilarity relationships. Pairwise relationships, also referred to as pairwise constraints, generally contain less information than class labels, but, in some contexts, are easier to obtain from human supervisors. The SNN architecture guarantees the basic properties of a similarity measure (symmetry and non negativity) and it can deal with non-transitivity of the similarity criterion. Unlike the majority of the metric learning algorithms proposed so far, it can model non-linear relationships among data still providing a natural out-of-sample extension to novel pairs of patterns. The theoretical properties of SNNs and their application to Semi-Supervised Clustering are investigated. In particular, we introduce a novel technique that allows the clustering algorithm to compute the optimal representatives of a data partition by means of backpropagation on the input layer, biased by a L-2 norm regularizer. An extensive set of experimental results are provided to compare SNNs with the most popular similarity learning algorithms. Both on benchmarks and real world data, SNNs and SNN-based clustering show improved performances, assessing the advantage of the proposed neural network approach to similarity measure learning. (c) 2011 Elsevier Ltd. All rights reserved.