Divergence-Based Vector Quantization

被引:53
作者
Villmann, Thomas [1 ]
Haase, Sven [1 ]
机构
[1] Univ Appl Sci Mittweida, Dept Math Nat & Comp Sci, D-09648 Mittweida, Germany
关键词
FUZZY CLASSIFICATION; INFORMATION; ROBUST;
D O I
10.1162/NECO_a_00110
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Supervised and unsupervised vector quantization methods for classification and clustering traditionally use dissimilarities, frequently taken as Euclidean distances. In this article, we investigate the applicability of divergences instead, focusing on online learning. We deduce the mathematical fundamentals for its utilization in gradient-based online vector quantization algorithms. It bears on the generalized derivatives of the divergences known as Frechet derivatives in functional analysis, which reduces in finite-dimensional problems to partial derivatives in a natural way. We demonstrate the application of this methodology for widely applied supervised and unsupervised online vector quantization schemes, including self-organizing maps, neural gas, and learning vector quantization. Additionally, principles for hyperparameter optimization and relevance learning for parameterized divergences in the case of supervised vector quantization are given to achieve improved classification accuracy.
引用
收藏
页码:1343 / 1392
页数:50
相关论文
共 71 条
[41]  
Liese F., 1987, Convex Statistical Distances
[42]   On divergences and informations in statistics and information theory [J].
Liese, Friedrich ;
Vajda, Igor .
IEEE TRANSACTIONS ON INFORMATION THEORY, 2006, 52 (10) :4394-4412
[43]   ALGORITHM FOR VECTOR QUANTIZER DESIGN [J].
LINDE, Y ;
BUZO, A ;
GRAY, RM .
IEEE TRANSACTIONS ON COMMUNICATIONS, 1980, 28 (01) :84-95
[44]   NEURAL-GAS NETWORK FOR VECTOR QUANTIZATION AND ITS APPLICATION TO TIME-SERIES PREDICTION [J].
MARTINETZ, TM ;
BERKOVICH, SG ;
SCHULTEN, KJ .
IEEE TRANSACTIONS ON NEURAL NETWORKS, 1993, 4 (04) :558-569
[45]  
MINIKA T, 2005, 173 MICR RES
[46]  
Mwebaze E., 2010, Proceedings of the 18th European Symposium on Artificial Neural Networks (ESANN 2010), P247
[47]   Sided and Symmetrized Bregman Centroids [J].
Nielsen, Frank ;
Nock, Richard .
IEEE TRANSACTIONS ON INFORMATION THEORY, 2009, 55 (06) :2882-2904
[48]  
Principe J.C., 2000, Unsupervised Adapt. Filter., V1, P265
[49]  
Qiao Y, 2008, INTERSPEECH 2008: 9TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION 2008, VOLS 1-5, P1349
[50]  
Ramsay J., 2006, Functional data analysis