A new regularized restricted Boltzmann machine based on class preserving

被引:11
作者
Hu, Junying [1 ]
Zhang, Jiangshe [1 ]
Ji, Nannan [2 ]
Zhang, Chunxia [1 ]
机构
[1] Xi An Jiao Tong Univ, Sch Math & Stat, Xian 710049, Peoples R China
[2] Changan Univ, Dept Math & Informat Sci, Xian 710054, Peoples R China
基金
中国国家自然科学基金;
关键词
Restricted Boltzmann machine; Feature learning; Class preserving; TRANSITION;
D O I
10.1016/j.knosys.2017.02.012
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
It is known that an Restricted Boltzmann machine (RBM) can be used as a feature extractor to automatically extract data features in a completely unsupervised learning manner. In this paper, we develop a new regularized RBM by adding the class information, referred to as class preserving RBM (CPr-RBM). Specifically, we impose two constraints on RBM to make the class information clearly reflected in extracted features. One constraint can decrease the distance between the features of the same class and the other one can increase the distance between the features of different classes. The two constraints introduce class information to RBM and make the extracted features contain more category information which contributes to a better classification result. Experiments are conducted on MNIST dataset and 20-newgroup dataset, which show that CPr-RBM learns more discriminate representations and outperforms other related state-of-the-art models in dealing with classification problems. (C) 2017 Elsevier B.V. All rights reserved.
引用
收藏
页码:1 / 12
页数:12
相关论文
共 23 条
[1]  
[Anonymous], 2005, AISTATS BRIDGETOWN B
[2]  
[Anonymous], 2010, P 13 INT C ARTIFICIA
[3]  
[Anonymous], 2008, P INT C MACH LEARN I
[4]  
[Anonymous], 2009, Advances in neural information processing systems
[5]  
[Anonymous], 2010, 2010 INT JOINT C NEU
[6]  
[Anonymous], 2009, P 26 ANN INT C MACH, DOI [DOI 10.1145/1553374.1553439, 10.1145/1553374.1553439]
[7]  
Belkin M, 2006, J MACH LEARN RES, V7, P2399
[8]   The flip-the-state transition operator for restricted Boltzmann machines [J].
Brugge, Kai ;
Fischer, Asja ;
Igel, Christian .
MACHINE LEARNING, 2013, 93 (01) :53-69
[9]  
CORTES C, 1995, MACH LEARN, V20, P273, DOI 10.1023/A:1022627411411
[10]  
Cover T. M., 2012, ELEMENTS INFORM THEO