Deep learning-based point-scanning super-resolution imaging

被引：108

作者：

Fang, Linjing ^{[1
]}

Monroe, Fred ^{[2
]}

Novak, Sammy Weiser ^{[1
]}

Kirk, Lyndsey ^{[3
]}

Schiavon, Cara R. ^{[1
]}

Yu, Seungyoon B. ^{[4
]}

Zhang, Tong ^{[1
]}

Wu, Melissa ^{[1
]}

Kastner, Kyle ^{[5
]}

Latif, Alaa Abdel ^{[6
]}

Lin, Zijun ^{[6
]}

Shaw, Andrew ^{[6
]}

Kubota, Yoshiyuki ^{[7
]}

Mendenhall, John ^{[3
]}

Zhang, Zhao ^{[8
]}

Pekkurnaz, Gulcin ^{[4
]}

Harris, Kristen ^{[3
]}

Howard, Jeremy ^{[6
]}

Manor, Uri ^{[1
]}

机构：

[1] Salk Inst Biol Studies, Waitt Adv Biophoton Ctr, 10010 N Torrey Pines Rd, La Jolla, CA 92037 USA

[2] Wicklow AI Med Res Initiat, San Francisco, CA USA

[3] Univ Texas Austin, Ctr Learning & Memory, Inst Neurosci, Dept Neurosci, Austin, TX 78712 USA

[4] Univ Calif San Diego, Neurobiol Sect, Div Biol Sci, La Jolla, CA 92093 USA

[5] Univ Montreal, Montreal Inst Learning Algorithms, Montreal, PQ, Canada

[6] Univ San Francisco, Data Inst, Fast AI, San Francisco, CA USA

[7] Natl Inst Physiol Sci, Div Cerebral Circuitry, Okazaki, Aichi, Japan

[8] Univ Texas Austin, Texas Adv Comp Ctr, Austin, TX 78712 USA

来源：

NATURE METHODS | 2021年 / 18卷 / 04期

基金：

美国国家科学基金会; 日本学术振兴会; 美国国家卫生研究院;

关键词：

RESTORATION; MICROSCOPY;

D O I：

10.1038/s41592-021-01080-z

中图分类号：

Q5 [生物化学];

学科分类号：

071010 ; 081704 ;

摘要：

Point-scanning imaging systems are among the most widely used tools for high-resolution cellular and tissue imaging, benefiting from arbitrarily defined pixel sizes. The resolution, speed, sample preservation and signal-to-noise ratio (SNR) of point-scanning systems are difficult to optimize simultaneously. We show these limitations can be mitigated via the use of deep learning-based supersampling of undersampled images acquired on a point-scanning system, which we term point-scanning super-resolution (PSSR) imaging. We designed a 'crappifier' that computationally degrades high SNR, high-pixel resolution ground truth images to simulate low SNR, low-resolution counterparts for training PSSR models that can restore real-world undersampled images. For high spatiotemporal resolution fluorescence time-lapse data, we developed a 'multi-frame' PSSR approach that uses information in adjacent frames to improve model predictions. PSSR facilitates point-scanning image acquisition with otherwise unattainable resolution, speed and sensitivity. All the training data, models and code for PSSR are publicly available at 3DEM.org.

引用

页码：406 / +

页数：27

共 57 条

[41]

Royer L.A, 2020, IMAGE DECONVOLUTION

[42] Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network [J].

Shi, Wenzhe ;

Caballero, Jose ;

Huszar, Ferenc ;

Totz, Johannes ;

Aitken, Andrew P. ;

Bishop, Rob ;

Rueckert, Daniel ;

Wang, Zehan .

2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2016, :1874-1883

[43] Learning from Simulated and Unsupervised Images through Adversarial Training [J].

Shrivastava, Ashish ;

Pfister, Tomas ;

Tuzel, Oncel ;

Susskind, Josh ;

Wang, Wenda ;

Webb, Russ .

30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, :2242-2251

[44]

Smith L. N., 2018, ARXIV

[45] Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates [J].

Smith, Leslie N. ;

Topin, Nicholay .

ARTIFICIAL INTELLIGENCE AND MACHINE LEARNING FOR MULTI-DOMAIN OPERATIONS APPLICATIONS, 2019, 11006

[46] Cyclical Learning Rates for Training Neural Networks [J].

Smith, Leslie N. .

2017 IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV 2017), 2017, :464-472

[47]

Sreehari S, 2017, P IEEE C COMP VIS PA, P88

[48]

Sugawara Y, 2018, IEEE IMAGE PROC, P66, DOI 10.1109/ICIP.2018.8451141

[49] Synaptic circuits and their variations within different columns in the visual system of Drosophila [J].

Takemura, Shin-ya ;

Xu, C. Shan ;

Lu, Zhiyuan ;

Rivlin, Patricia K. ;

Parag, Toufiq ;

Olbris, Donald J. ;

Plaza, Stephen ;

Zhao, Ting ;

Katz, William T. ;

Umayam, Lowell ;

Weaver, Charlotte ;

Hess, Harald F. ;

Horne, Jane Anne ;

Nunez-Iglesias, Juan ;

Aniceto, Roxanne ;

Chang, Lei-Ann ;

Lauchie, Shirley ;

Nasca, Ashley ;

Ogundeyi, Omotara ;

Sigmund, Christopher ;

Takemura, Satoko ;

Tran, Julie ;

Langille, Carlie ;

Le Lacheur, Kelsey ;

McLin, Sari ;

Shinomiya, Aya ;

Chklovskii, Dmitri B. ;

Meinertzhagen, Ian A. ;

Scheffer, Louis K. .

PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA, 2015, 112 (44) :13711-13716

[50] Artificial intelligence for microscopy: what you should know [J].

von Chamier, Lucas ;

Laine, Romain F. ;

Henriques, Ricardo .

BIOCHEMICAL SOCIETY TRANSACTIONS, 2019, 47 :1029-1040

← 1 2 3 4 5 6 →