Principal Component Analysis with Noisy and/or Missing Data

被引：98

作者：

Bailey, Stephen ^{[1
]}

机构：

[1] Univ Calif Berkeley, Lawrence Berkeley Natl Lab, Div Phys, Berkeley, CA 94720 USA

来源：

PUBLICATIONS OF THE ASTRONOMICAL SOCIETY OF THE PACIFIC | 2012年 / 124卷 / 919期

基金：

美国国家科学基金会; 美国国家航空航天局;

关键词：

MAXIMUM-LIKELIHOOD; INCOMPLETE DATA; EM ALGORITHM; SPECTRA;

D O I：

10.1086/668105

中图分类号：

P1 [天文学];

学科分类号：

0704 ;

摘要：

We present a method for performing principal component analysis (PCA) on noisy datasets with missing values. Estimates of the measurement error are used to weight the input data such that the resulting eigenvectors, when compared to classic PCA, are more sensitive to the true underlying signal variations rather than being pulled by heteroskedastic measurement noise. Missing data are simply limiting cases of weight = 0. The underlying algorithm is a noise weighted expectation maximization (EM) PCA, which has additional benefits of implementation speed and flexibility for smoothing eigenvectors to reduce the noise contribution. We present applications of this method on simulated data and QSO spectra from the Sloan Digital Sky Survey (SDSS).

引用

页码：1015 / 1023

页数：9

共 25 条

[1] THE SEVENTH DATA RELEASE OF THE SLOAN DIGITAL SKY SURVEY [J].

Abazajian, Kevork N. ;

Adelman-McCarthy, Jennifer K. ;

Agueros, Marcel A. ;

Allam, Sahar S. ;

Prieto, Carlos Allende ;

An, Deokkeun ;

Anderson, Kurt S. J. ;

Anderson, Scott F. ;

Annis, James ;

Bahcall, Neta A. ;

Bailer-Jones, C. A. L. ;

Barentine, J. C. ;

Bassett, Bruce A. ;

Becker, Andrew C. ;

Beers, Timothy C. ;

Bell, Eric F. ;

Belokurov, Vasily ;

Berlind, Andreas A. ;

Berman, Eileen F. ;

Bernardi, Mariangela ;

Bickerton, Steven J. ;

Bizyaev, Dmitry ;

Blakeslee, John P. ;

Blanton, Michael R. ;

Bochanski, John J. ;

Boroski, William N. ;

Brewington, Howard J. ;

Brinchmann, Jarle ;

Brinkmann, J. ;

Brunner, Robert J. ;

Budavari, Tamas ;

Carey, Larry N. ;

Carliles, Samuel ;

Carr, Michael A. ;

Castander, Francisco J. ;

Cinabro, David ;

Connolly, A. J. ;

Csabai, Istvan ;

Cunha, Carlos E. ;

Czarapata, Paul C. ;

Davenport, James R. A. ;

de Haas, Ernst ;

Dilday, Ben ;

Doi, Mamoru ;

Eisenstein, Daniel J. ;

Evans, Michael L. ;

Evans, N. W. ;

Fan, Xiaohui ;

Friedman, Scott D. ;

Frieman, Joshua A. .

ASTROPHYSICAL JOURNAL SUPPLEMENT SERIES, 2009, 182 (02) :543-558

[2]

[Anonymous], 2002, Principal components analysis

[3] DOMAINS OF CONVERGENCE FOR THE EM ALGORITHM - A CAUTIONARY TALE IN A LOCATION ESTIMATION PROBLEM [J].

ARSLAN, O ;

CONSTABLE, PDL ;

KENT, JT .

STATISTICS AND COMPUTING, 1993, 3 (03) :103-108

[4] K-corrections and filter transformations in the ultraviolet, optical, and near-infrared [J].

Blanton, Michael R. ;

Roweis, Sam .

ASTRONOMICAL JOURNAL, 2007, 133 (02) :734-754

[5] SIGNAL-TO-NOISE EIGENMODE ANALYSIS OF THE 2-YEAR COBE MAPS [J].

BOND, JR .

PHYSICAL REVIEW LETTERS, 1995, 74 (22) :4369-4372

[6] A robust classification of galaxy spectra: Dealing with noisy and incomplete data [J].

Connolly, AJ ;

Szalay, AS .

ASTRONOMICAL JOURNAL, 1999, 117 (05) :2052-2062

[7] MAXIMUM LIKELIHOOD FROM INCOMPLETE DATA VIA EM ALGORITHM [J].

DEMPSTER, AP ;

LAIRD, NM ;

RUBIN, DB .

JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-METHODOLOGICAL, 1977, 39 (01) :1-38

[8] LOWER RANK APPROXIMATION OF MATRICES BY LEAST-SQUARES WITH ANY CHOICE OF WEIGHTS [J].

GABRIEL, KR ;

ZAMIR, S .

TECHNOMETRICS, 1979, 21 (04) :489-498

[9] Improved redshifts for SDSS quasar spectra [J].

Hewett, Paul C. ;

Wild, Vivienne .

MONTHLY NOTICES OF THE ROYAL ASTRONOMICAL SOCIETY, 2010, 405 (04) :2302-2316

[10] Analysis of a complex of statistical variables into principal components [J].

Hotelling, H .

JOURNAL OF EDUCATIONAL PSYCHOLOGY, 1933, 24 :417-441

← 1 2 3 →