Fast model-based clustering of partial records

被引:3
作者
Goren, Emily M. [1 ,2 ]
Maitra, Ranjan [1 ]
机构
[1] Iowa State Univ, Dept Stat, Ames, IA 50011 USA
[2] Seagen, 21823 30th Dr SE, Bothell, WA 98021 USA
基金
美国食品与农业研究所;
关键词
finite mixture models; imputation; modified em-EM algorithm; Rnd-EM algorithm; unsupervised learning; GAMMA-RAY BURSTS; T MIXTURE-MODELS; SIMULATING DATA; INCOMPLETE DATA; MISSING VALUES; EM ALGORITHM; IMPUTATION; LIKELIHOOD; CLASSIFICATION; PERFORMANCE;
D O I
10.1002/sta4.416
中图分类号
O21 [概率论与数理统计]; C8 [统计学];
学科分类号
020208 ; 070103 ; 0714 ;
摘要
Partially recorded data are frequently encountered in many applications and usually clustered by first removing incomplete cases or features with missing values, or by imputing missing values, followed by application of a clustering algorithm to the resulting altered dataset. Here, we develop clustering methodology through a model-based approach using the marginal density for the observed values, assuming a finite mixture model of multivariate t distributions. We compare our approximate algorithm to the corresponding full expectation-maximization (EM) approach that considers missing values in the incomplete dataset and makes a missing at random (MAR) assumption, as well as case deletion and imputation methods. Since only the observed values are utilized, our approach is computationally more efficient than imputation or full EM. Simulation studies demonstrate that our approach has favourable recovery of the true cluster partition compared to case deletion and imputation under various missingness mechanisms and is at least competitive with the full EM approach, even when MAR assumptions are violated. Our methodology is demonstrated on a problem of clustering gamma-ray bursts and is implemented at https://github.com/emilygoren/MixtClust.
引用
收藏
页数:20
相关论文
共 85 条
[52]   CATALOG OF COSMIC GAMMA-RAY BURSTS FROM THE KONUS EXPERIMENT DATA .1.2. [J].
MAZETS, EP ;
GOLENETSKII, SV ;
ILINSKII, VN ;
PANOV, VN ;
APTEKAR, RL ;
GURYAN, YA ;
PROSKURA, MP ;
SOKOLOV, IA ;
SOKOLOVA, ZY ;
KHARITONOVA, TV ;
DYATCHKOV, AV ;
KHAVENSON, NG .
ASTROPHYSICS AND SPACE SCIENCE, 1981, 80 (01) :3-83
[53]  
McLachlan G. J., 1998, Advances in Pattern Recognition. Joint IAPR International Workshops SSPR'98 and SPR'98. Proceedings, P658, DOI 10.1007/BFb0033290
[54]  
McLachlan GJ, 2008, WILEY SER PROBAB ST, P365
[55]  
McNicholas PD., 2016, MIXTURE MODEL BASED, DOI 10.1201/9781315373577
[56]  
Melnykov V, 2012, J STAT SOFTW, V51, P1
[57]   Finite mixture models and model-based clusteringFinite mixture models and model-based clustering [J].
Melnykov, Volodymyr ;
Maitra, Ranjan .
STATISTICS SURVEYS, 2010, 4 :80-116
[58]  
Melnykov V, 2011, J MACH LEARN RES, V12, P69
[59]   The EM algorithm - An old folk-song sung to a fast new tune [J].
Meng, XL ;
vanDyk, D .
JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-METHODOLOGICAL, 1997, 59 (03) :511-540
[60]   Three types of gamma-ray bursts [J].
Mukherjee, S ;
Feigelson, ED ;
Babu, GJ ;
Murtagh, F ;
Fraley, C ;
Raftery, A .
ASTROPHYSICAL JOURNAL, 1998, 508 (01) :314-327