Information-Maximization Clustering Based on Squared-Loss Mutual Information

被引:27
作者
Sugiyama, Masashi [1 ]
Niu, Gang [1 ]
Yamada, Makoto [2 ]
Kimura, Manabu [1 ]
Hachiya, Hirotaka [1 ]
机构
[1] Tokyo Inst Technol, Merugo Ku, Tokyo 1528552, Japan
[2] Yahoo Labs, Sunnyvale, CA 94089 USA
关键词
VARIATIONAL INFERENCE; MEAN-SHIFT; K-MEANS; MIXTURES; HARDNESS;
D O I
10.1162/NECO_a_00534
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Information-maximization clustering learns a probabilistic classifier in an unsupervised manner so that mutual information between feature vectors and cluster assignments is maximized. A notable advantage of this approach is that it involves only continuous optimization of model parameters, which is substantially simpler than discrete optimization of cluster assignments. However, existing methods still involve nonconvex optimization problems, and therefore finding a good local optimal solution is not straightforward in practice. In this letter, we propose an alternative information-maximization clustering method based on a squared-loss variant of mutual information. This novel approach gives a clustering solution analytically in a computationally efficient way via kernel eigenvalue decomposition. Furthermore, we provide a practical model selection procedure that allows us to objectively optimize tuning parameters included in the kernel function. Through experiments, we demonstrate the usefulness of the proposed approach.
引用
收藏
页码:84 / 131
页数:48
相关论文
共 32 条
[21]   Clustering of web search results based on the cuckoo search algorithm and Balanced Bayesian Information Criterion [J].
Cobos, Carlos ;
Munoz-Collazos, Henry ;
Urbano-Munoz, Richar ;
Mendoza, Martha ;
Leon, Elizabeth ;
Herrera-Viedma, Enrique .
INFORMATION SCIENCES, 2014, 281 :248-264
[22]   MIC-KMeans: A Maximum Information Coefficient Based High-Dimensional Clustering Algorithm [J].
Wang, Ruping ;
Li, Hui ;
Chen, Mei ;
Dai, Zhenyu ;
Zhu, Ming .
ARTIFICIAL INTELLIGENCE AND ALGORITHMS IN INTELLIGENT SYSTEMS, 2019, 764 :208-218
[23]   Towards unveiling individual differences in different stages of information processing: a clustering-based approach [J].
Hofmans, Joeri ;
Mullet, Etienne .
QUALITY & QUANTITY, 2013, 47 (01) :455-464
[24]   A SPA-BASED K-MEANS CLUSTERING ALGORITHM FOR THE REMOTE SENSING INFORMATION EXTRACTION [J].
Xie, Xiangjian ;
Zhao, Junsan ;
Li, Hongbo ;
Zhang, Wanqiang ;
Yuan, Lei .
2012 IEEE INTERNATIONAL GEOSCIENCE AND REMOTE SENSING SYMPOSIUM (IGARSS), 2012, :6111-6114
[25]   Towards unveiling individual differences in different stages of information processing: a clustering-based approach [J].
Joeri Hofmans ;
Etienne Mullet .
Quality & Quantity, 2013, 47 :455-464
[26]   Three-way clustering method for incomplete information system based on set-pair analysis [J].
Zhang, Chunying ;
Gao, Ruiyan ;
Qin, Hao ;
Feng, Xiaoze .
GRANULAR COMPUTING, 2021, 6 (02) :389-398
[27]   A Novel Approach to Air Passenger Index Prediction: Based on Mutual Information Principle and Support Vector Regression Blended Mode [J].
Xiong, Honglin ;
Fan, Chongjun ;
Chen, Hongmin ;
Yang, Yun ;
Antwi, Collins Opoku ;
Fan, Xiaomao .
SAGE OPEN, 2022, 12 (01)
[28]   Improvement and application of information communication technology in wireless routing protocol based on adaptive K-means clustering algorithm [J].
Chen, Zili ;
Pan, Yanyan .
WIRELESS NETWORKS, 2024, 30 (06) :5997-6009
[29]   A k-Means-Like Algorithm for Clustering Categorical Data Using an Information Theoretic-Based Dissimilarity Measure [J].
Thu-Hien Thi Nguyen ;
Van-Nam Huynh .
FOUNDATIONS OF INFORMATION AND KNOWLEDGE SYSTEMS (FOIKS 2016), 2016, 9616 :115-130
[30]   ON THE USE OF CONTEXTUAL TIME-FREQUENCY INFORMATION FOR FULL-BAND CLUSTERING-BASED CONVOLUTIVE BLIND SOURCE SEPARATION [J].
Atcheson, Matt ;
Jafari, Ingrid ;
Togneri, Roberto ;
Nordholm, Sven .
2014 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2014,