On size-independent sample complexity of ReLU networks

被引:0
|
作者
Sellke, Mark [1 ]
机构
[1] Harvard Stat, Cambridge, MA USA
关键词
Neural networks; Rademacher complexity; Generalization; Theory of computation;
D O I
10.1016/j.ipl.2024.106482
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
We study the sample complexity of learning ReLU neural networks from the point of view of generalization. Given norm constraints on the weight matrices, a common approach is to estimate the Rademacher complexity of the associated function class. Previously [9] obtained a bound independent of the network size (scaling with a product of Frobenius norms) except for a factor of the square -root depth. We give a refinement which often has no explicit depth -dependence at all.
引用
收藏
页数:3
相关论文
共 50 条
  • [1] Size-independent sample complexity of neural networks
    Golowich, Noah
    Rakhlin, Alexander
    Shamir, Ohad
    INFORMATION AND INFERENCE-A JOURNAL OF THE IMA, 2020, 9 (02) : 473 - 504
  • [2] The size-independent oxygen cost of running
    McCann, DJ
    Adams, WC
    MEDICINE AND SCIENCE IN SPORTS AND EXERCISE, 2003, 35 (06): : 1049 - 1056
  • [3] Size-Independent Cylindrical Wheel Resonator
    Uslenghi, Piergiorgio L. E.
    IEEE ANTENNAS AND WIRELESS PROPAGATION LETTERS, 2013, 12 : 655 - 656
  • [4] Measuring the Size-Independent Fracture Energy of Concrete
    Cifuentes, H.
    Alcalde, M.
    Medina, F.
    STRAIN, 2013, 49 (01) : 54 - 59
  • [5] Size-independent scaling analysis for explosive percolation
    Hagiwara, Kenta
    Ozeki, Yukiyasu
    PHYSICAL REVIEW E, 2022, 106 (05)
  • [6] Subspace Surface Hopping with Size-Independent Dynamics
    Qiu, Jing
    Bai, Xin
    Wang, Linjun
    JOURNAL OF PHYSICAL CHEMISTRY LETTERS, 2019, 10 (03): : 637 - +
  • [7] Luminescent Gold Nanoparticles with Size-Independent Emission
    Liu, Jinbin
    Duchesne, Paul N.
    Yu, Mengxiao
    Jiang, Xingya
    Ning, Xuhui
    Vinluan, Rodrigo D., III
    Zhang, Peng
    Zheng, Jie
    ANGEWANDTE CHEMIE-INTERNATIONAL EDITION, 2016, 55 (31) : 8894 - 8898
  • [8] "Size-Independent" Single-Electron Tunneling
    Zhao, Jianli
    Sun, Shasha
    Swartz, Logan
    Riechers, Shawn
    Hu, Peiguang
    Chen, Shaowei
    Zheng, Jie
    Liu, Gang-yu
    JOURNAL OF PHYSICAL CHEMISTRY LETTERS, 2015, 6 (24): : 4986 - 4990
  • [9] Simple Size-Independent Measure of Ligand Efficiency
    Nissink, J. Willem M.
    JOURNAL OF CHEMICAL INFORMATION AND MODELING, 2009, 49 (06) : 1617 - 1622
  • [10] Liquid metasurface for size-independent detection of microplastics
    Zhao, Mingfu
    Chen, Feng
    Zhang, Bin
    Liu, Hong
    Li, Zeying
    Li, Gengchen
    Zhao, Minggang
    Ma, Ye
    TALANTA, 2025, 284