Optimally regularised kernel Fisher discriminant classification

被引:18
|
作者
Saadi, Kamel [1 ]
Talbot, Nicola L. C. [1 ]
Cawley, Gavin C. [1 ]
机构
[1] Univ E Anglia, Sch Comp Sci, Norwich NR4 7TJ, Norfolk, England
基金
英国生物技术与生命科学研究理事会;
关键词
model selection; cross-validation; least-squares support vector machine;
D O I
10.1016/j.neunet.2007.05.005
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Mika, Ratsch, Weston, Scholkopf and Muller [Mika, S., Ratsch, G.. Weston, J., Scholkopf, B., & Muller, K.-R. (1999). Fisher discriminant analysis with kernels. In Neural networks for signal processing: Vol. IX (pp. 41-48). New York: IEEE Press] introduce a non-linear formulation of Fisher's linear discriminant, based on the now familiar "kernel trick", demonstrating state-of-the-art performance on a wide range of real-world benchmark datasets. In this paper, we extend an existing analytical expression for the leave-one-out cross-validation error [Cawley, G. C., & Talbot, N. L. C. (2003b). Efficient leave-one-out cross-validation of kernel Fisher discriminant classifiers. Pattern Recognition, 36(11), 2585-2592] such that the leave-one-out error can be re-estimated following a change in the value of the regularisation parameter with a computational complexity of only O(l(2)) operations, which is substantially less than the O(l(3)) operations required for the basic training algorithm. This allows the regularisation parameter to be tuned at an essentially negligible computational cost. This is achieved by performing the discriminant analysis in canonical form. The proposed method is therefore a useful component of a model selection strategy for this class of kernel machines that alternates between updates of the kernel and regularisation parameters. Results obtained on real-world and synthetic benchmark datasets indicate that the proposed method is competitive with model selection based on k-fold cross-validation in terms of generalisation, whilst being considerably faster. (C) 2007 Elsevier Ltd. All rights reserved.
引用
收藏
页码:832 / 841
页数:10
相关论文
共 50 条
  • [21] Kernel fisher discriminant analysis for palmprint recognition
    Wang, Yanxia
    Ruan, Qiuqi
    18TH INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION, VOL 4, PROCEEDINGS, 2006, : 457 - +
  • [22] Fisher plus Kernel criterion for discriminant analysis
    Yang, S
    Yan, SC
    Xu, D
    Tang, XO
    Zhang, C
    2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol 2, Proceedings, 2005, : 197 - 202
  • [23] Large Scale Indefinite Kernel Fisher Discriminant
    Schleif, Frank-Michael
    Gisbrecht, Andrej
    Tino, Peter
    SIMILARITY-BASED PATTERN RECOGNITION, SIMBAD 2015, 2015, 9370 : 160 - 170
  • [24] Properties of Standard and Sketched Kernel Fisher Discriminant
    Liu, Jiamin
    Xu, Wangli
    Zhang, Fode
    Lian, Heng
    IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2023, 45 (08) : 10596 - 10602
  • [25] A kernel Fisher discriminant classifier for speaker recognition
    Li, X.
    Zheng, Y.
    DCABES 2006 PROCEEDINGS, VOLS 1 AND 2, 2006, : 344 - 348
  • [26] Binary tree support vector machine based on Kernel Fisher Discriminant for multi-classification
    Liu, Bo
    Yang, Xiaowei
    Hao, Zhifeng
    ADVANCES IN NEURAL NETWORKS - ISNN 2006, PT 1, 2006, 3971 : 997 - 1003
  • [27] FEATURE SELECTION USING KERNEL BASED LOCAL FISHER DISCRIMINANT ANALYSIS FOR HYPERSPECTRAL IMAGE CLASSIFICATION
    Zhang, Guangyun
    Jia, Xiuping
    2011 IEEE INTERNATIONAL GEOSCIENCE AND REMOTE SENSING SYMPOSIUM (IGARSS), 2011, : 1728 - 1731
  • [28] Individualized learning for improving kernel Fisher discriminant analysis
    Fan, Zizhu
    Xu, Yong
    Ni, Ming
    Fang, Xiaozhao
    Zhang, David
    PATTERN RECOGNITION, 2016, 58 : 100 - 109
  • [29] Fast algorithm about kernel fisher discriminant analysis
    Zhao, Feng
    Zhang, Jun-Ying
    Liang, Jun-Li
    Dianzi Yu Xinxi Xuebao/Journal of Electronics and Information Technology, 2007, 29 (07): : 1731 - 1734
  • [30] Face annotation using Transductive Kernel Fisher Discriminant
    Zhu, Jianke
    Hoi, Steven C. H.
    Lyu, Michael R.
    IEEE TRANSACTIONS ON MULTIMEDIA, 2008, 10 (01) : 86 - 96