Multinomial random forest

被引:59
作者
Bai, Jiawang [1 ,2 ]
Li, Yiming [1 ]
Li, Jiawei [1 ]
Yang, Xue [1 ,2 ]
Jiang, Yong [1 ,2 ]
Xia, Shu-Tao [1 ,2 ]
机构
[1] Tsinghua Univ, Tsinghua Shenzhen Int Grad Sch, Shenzhen, Peoples R China
[2] Peng Cheng Lab, PCL Res Ctr Networks & Commun, Shenzhen, Peoples R China
基金
中国博士后科学基金; 中国国家自然科学基金;
关键词
Random forest; Consistency; Differential privacy; Classification; CONSISTENCY; ENSEMBLE;
D O I
10.1016/j.patcog.2021.108331
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Despite the impressive performance of random forests (RF), its theoretical properties have not been thoroughly understood. In this paper, we propose a novel RF framework, dubbed multinomial random forest (MRF), to analyze its consistency and privacy-preservation . Instead of deterministic greedy split rule or with simple randomness, the MRF adopts two impurity-based multinomial distributions to randomly select a splitting feature and a splitting value, respectively. Theoretically, we prove the consistency of MRF and analyze its privacy-preservation within the framework of differential privacy. We also demonstrate with multiple datasets that its performance is on par with the standard RF. To the best of our knowledge, MRF is the first consistent RF variant that has comparable performance to the standard RF. The code is available at https://github.com/jiawangbai/Multinomial- Random-Forest . (c) 2021 Published by Elsevier Ltd.
引用
收藏
页数:13
相关论文
共 46 条
  • [1] Face description with local binary patterns:: Application to face recognition
    Ahonen, Timo
    Hadid, Abdenour
    Pietikainen, Matti
    [J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2006, 28 (12) : 2037 - 2041
  • [2] [Anonymous], 2013, PROBABILISTIC THEORY
  • [3] GENERALIZED RANDOM FORESTS
    Athey, Susan
    Tibshirani, Julie
    Wager, Stefan
    [J]. ANNALS OF STATISTICS, 2019, 47 (02) : 1148 - 1178
  • [4] Biau G, 2012, J MACH LEARN RES, V13, P1063
  • [5] Biau G, 2008, J MACH LEARN RES, V9, P2015
  • [6] Bifet A, 2009, KDD-09: 15TH ACM SIGKDD CONFERENCE ON KNOWLEDGE DISCOVERY AND DATA MINING, P139
  • [7] Breiman L., 2017, CLASSIFICATION REGRE, DOI DOI 10.1201/9781315139470-8
  • [8] Breiman L., 2001, Machine Learning, V45, P5
  • [9] Breiman L., 2004, CONSISTENCY SIMPLE M
  • [10] Cootes TF, 2012, LECT NOTES COMPUT SC, V7578, P278, DOI 10.1007/978-3-642-33786-4_21