Learning multiple linear manifolds with self-organizing networks

Learning multiple linear manifolds permits one to deal with multiple variations incurred in target objects. The adaptive-subspace self-organizing map (ASSOM) proposed by Kohonen can learn a set of ordered subspaces, i.e. linear manifolds passing through the origin, but not those shifted away from the origin. The linear manifold self-organizing map (LMSOM) proposed in this paper considers offsets of linear manifolds from the origin and aims to learn linear manifolds by minimizing a projection error function in a gradient-descent fashion. At each learning step, the winning module and its neighbours update basis vectors as well as offset vectors of their manifolds towards the negative gradient of the error function. Experiments show that the LMSOM can learn clusters aligned on linear manifolds shifted away from the origin and separate them accordingly. The LMSOM is applied to handwritten digit recognition and shows promising results.