Kernel Discriminant Analysis for Positive Definite and Indefinite Kernels

被引:81
作者
Pekalska, Elzbieta [1 ]
Haasdonk, Bernard [2 ]
机构
[1] Univ Manchester, Sch Comp Sci, Manchester M13 9PL, Lancs, England
[2] Univ Stuttgart, Inst Appl Anal & Numer Simulat, D-70569 Stuttgart, Germany
基金
英国工程与自然科学研究理事会;
关键词
Machine learning; pattern recognition; kernel methods; indefinite kernels; discriminant analysis; CLASSIFICATION;
D O I
10.1109/TPAMI.2008.290
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Kernel methods are a class of well established and successful algorithms for pattern analysis due to their mathematical elegance and good performance. Numerous nonlinear extensions of pattern recognition techniques have been proposed so far based on the so-called kernel trick. The objective of this paper is twofold. First, we derive an additional kernel tool that is still missing, namely kernel quadratic discriminant(KQD). We discuss different formulations of KQD based on the regularized kernel Mahalanobis distance in both complete and class-related subspaces. Second, we propose suitable extensions of kernel linear and quadratic discriminants to indefinite kernels. We provide classifiers that are applicable to kernels defined by any symmetric similarity measure. This is important in practice because problem-suited proximity measures often violate the requirement of positive definiteness. As in the traditional case, KQD can be advantageous for data with unequal class spreads in the kernel-induced spaces, which cannot be well separated by a linear discriminant. We illustrate this on artificial and real data for both positive definite and indefinite kernels.
引用
收藏
页码:1017 / 1031
页数:15
相关论文
共 41 条
[1]  
[Anonymous], 2005, Kernel fisher's discriminant analysis in gaussian reproducing kernel hilbert space
[2]  
[Anonymous], 2005, The Dissimilarity Representation for Pattern Recognition
[3]  
[Anonymous], 2001, Pattern Classification
[4]  
Bognar J., 1974, Indefinite Inner Product Spaces
[5]  
CANU S, 2003, ADV LEARNING THEORY, V190, P89
[6]   LIBSVM: A Library for Support Vector Machines [J].
Chang, Chih-Chung ;
Lin, Chih-Jen .
ACM TRANSACTIONS ON INTELLIGENT SYSTEMS AND TECHNOLOGY, 2011, 2 (03)
[7]  
Der R., 2007, JMLR WORKSH C P, P91
[8]  
DRITSCHEL M, 1996, FIELDS I MONOGRAPHS, P141
[9]  
DUBUISSON MP, 1994, INT C PATT RECOG, P566, DOI 10.1109/ICPR.1994.576361
[10]  
Goldfarb L., 1985, Progress in Pattern Recognition, V2, P241