Residual Multiplicative Filter Networks for Multiscale Reconstruction

被引:0
作者
Shekarforoush, Shayan [1 ,2 ]
Lindell, David B. [1 ,2 ]
Fleet, David J. [1 ,2 ,3 ]
Brubaker, Marcus A. [1 ,2 ,4 ]
机构
[1] Univ Toronto, Toronto, ON, Canada
[2] Vector Inst, Toronto, ON, Canada
[3] Google Res, Mountain View, CA USA
[4] York Univ, N York, ON, Canada
来源
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 35 (NEURIPS 2022) | 2022年
基金
加拿大自然科学与工程研究理事会;
关键词
WAVELETS;
D O I
暂无
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Coordinate networks like Multiplicative Filter Networks (MFNs) and BACON offer some control over the frequency spectrum used to represent continuous signals such as images or 3D volumes. Yet, they are not readily applicable to problems for which coarse-to-fine estimation is required, including various inverse problems in which coarse-to-fine optimization plays a key role in avoiding poor local minima. We introduce a new coordinate network architecture and training scheme that enables coarse-to-fine optimization with fine-grained control over the frequency support of learned reconstructions. This is achieved with two key innovations. First, we incorporate skip connections so that structure at one scale is preserved when fitting finer-scale structure. Second, we propose a novel initialization scheme to provide control over the model frequency spectrum at each stage of optimization. We demonstrate how these modifications enable multiscale optimization for coarse-to-fine fitting to natural images. We then evaluate our model on synthetically generated datasets for the the problem of single-particle cryo-EM reconstruction. We learn high resolution multiscale structures, on par with the state-of-the art. Project webpage: https://shekshaa.github.io/ResidualMFN/.
引用
收藏
页数:14
相关论文
共 64 条
  • [1] Adelson E. H., 1984, RCA engineer, V29, P33
  • [2] Rapid Solution of the Cryo-EM Reconstruction Problem by Frequency Marching
    Barnett, Alex
    Greengard, Leslie
    Pataki, Andras
    Spivak, Marina
    [J]. SIAM JOURNAL ON IMAGING SCIENCES, 2017, 10 (03): : 1170 - 1195
  • [3] Barron Jonathan T, 2021, P ICCV
  • [4] Bramble J. H., 2019, MULTIGRID METHODS
  • [5] THE LAPLACIAN PYRAMID AS A COMPACT IMAGE CODE
    BURT, PJ
    ADELSON, EH
    [J]. IEEE TRANSACTIONS ON COMMUNICATIONS, 1983, 31 (04) : 532 - 540
  • [6] Chabra R., 2020, P EUR C COMP VIS, P608
  • [7] pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis
    Chan, Eric R.
    Monteiro, Marco
    Kellnhofer, Petr
    Wu, Jiajun
    Wetzstein, Gordon
    [J]. 2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, : 5795 - 5805
  • [8] Chan Eric R., 2022, P CVPR
  • [9] Deng Yu, 2022, P CVPR
  • [10] Fathony R., 2021, P ICLR