Stochastic Optimization for Multiview Representation Learning using Partial Least Squares

被引:0
作者
Arora, Raman [1 ]
Mianjy, Poorya [1 ]
Marinov, Teodor, V [2 ]
机构
[1] Johns Hopkins Univ, Dept Comp Sci, Baltimore, MD 21218 USA
[2] Univ Edinburgh, Sch Informat, Edinburgh EH8 9AB, Midlothian, Scotland
来源
INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 48 | 2016年 / 48卷
关键词
D O I
暂无
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Partial Least Squares (PLS) is a ubiquitous statistical technique for bilinear factor analysis. It is used in many data analysis, machine learning, and information retrieval applications to model the covariance structure between a pair of data matrices. In this paper, we consider PLS for representation learning in a multiview setting where we have more than one view in data at training time. Furthermore, instead of framing PLS as a problem about a fixed given data set, we argue that PLS should be studied as a stochastic optimization problem, especially in a "big data" setting, with the goal of optimizing a population objective based on sample. This view suggests using Stochastic Approximation (SA) approaches, such as Stochastic Gradient Descent (SGD) and enables a rigorous analysis of their benefits. In this paper, we develop SA approaches to PLS and provide iteration complexity bounds for the proposed algorithms.
引用
收藏
页数:9
相关论文
共 27 条
  • [1] [Anonymous], 2013, Advances in Neural Information Processing System
  • [2] [Anonymous], 2008, P 25 INT C MACHINE L, DOI [10.1145/1390156.1390273, DOI 10.1145/1390156.1390273]
  • [3] [Anonymous], 2016, ARXIV160201024
  • [4] [Anonymous], 2011, Advances in Neural Information Processing Systems
  • [5] Arora R., 2012, PROC MACHINE LEARNIN, P34
  • [6] Arora R., 2013, ICASSP
  • [7] Arora R, 2012, ANN ALLERTON CONF, P861, DOI 10.1109/Allerton.2012.6483308
  • [8] Benton Adrian, ACL
  • [9] Bharadwaj Sujeeth, 2012, WORKSH STAT MACH LEA
  • [10] Correlational spectral clustering
    Blaschko, Matthew B.
    Lampert, Christoph H.
    [J]. 2008 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, VOLS 1-12, 2008, : 93 - +