Deep learning-based point-scanning super-resolution imaging

被引：108

作者：

Fang, Linjing ^{[1
]}

Monroe, Fred ^{[2
]}

Novak, Sammy Weiser ^{[1
]}

Kirk, Lyndsey ^{[3
]}

Schiavon, Cara R. ^{[1
]}

Yu, Seungyoon B. ^{[4
]}

Zhang, Tong ^{[1
]}

Wu, Melissa ^{[1
]}

Kastner, Kyle ^{[5
]}

Latif, Alaa Abdel ^{[6
]}

Lin, Zijun ^{[6
]}

Shaw, Andrew ^{[6
]}

Kubota, Yoshiyuki ^{[7
]}

Mendenhall, John ^{[3
]}

Zhang, Zhao ^{[8
]}

Pekkurnaz, Gulcin ^{[4
]}

Harris, Kristen ^{[3
]}

Howard, Jeremy ^{[6
]}

Manor, Uri ^{[1
]}

机构：

[1] Salk Inst Biol Studies, Waitt Adv Biophoton Ctr, 10010 N Torrey Pines Rd, La Jolla, CA 92037 USA

[2] Wicklow AI Med Res Initiat, San Francisco, CA USA

[3] Univ Texas Austin, Ctr Learning & Memory, Inst Neurosci, Dept Neurosci, Austin, TX 78712 USA

[4] Univ Calif San Diego, Neurobiol Sect, Div Biol Sci, La Jolla, CA 92093 USA

[5] Univ Montreal, Montreal Inst Learning Algorithms, Montreal, PQ, Canada

[6] Univ San Francisco, Data Inst, Fast AI, San Francisco, CA USA

[7] Natl Inst Physiol Sci, Div Cerebral Circuitry, Okazaki, Aichi, Japan

[8] Univ Texas Austin, Texas Adv Comp Ctr, Austin, TX 78712 USA

来源：

NATURE METHODS | 2021年 / 18卷 / 04期

基金：

美国国家科学基金会; 日本学术振兴会; 美国国家卫生研究院;

关键词：

RESTORATION; MICROSCOPY;

D O I：

10.1038/s41592-021-01080-z

中图分类号：

Q5 [生物化学];

学科分类号：

071010 ; 081704 ;

摘要：

Point-scanning imaging systems are among the most widely used tools for high-resolution cellular and tissue imaging, benefiting from arbitrarily defined pixel sizes. The resolution, speed, sample preservation and signal-to-noise ratio (SNR) of point-scanning systems are difficult to optimize simultaneously. We show these limitations can be mitigated via the use of deep learning-based supersampling of undersampled images acquired on a point-scanning system, which we term point-scanning super-resolution (PSSR) imaging. We designed a 'crappifier' that computationally degrades high SNR, high-pixel resolution ground truth images to simulate low SNR, low-resolution counterparts for training PSSR models that can restore real-world undersampled images. For high spatiotemporal resolution fluorescence time-lapse data, we developed a 'multi-frame' PSSR approach that uses information in adjacent frames to improve model predictions. PSSR facilitates point-scanning image acquisition with otherwise unattainable resolution, speed and sensitivity. All the training data, models and code for PSSR are publicly available at 3DEM.org.

引用

页码：406 / +

页数：27

共 57 条

[1]

Aitken A, 2017, CHECKERBOARD ARTIFAC, DOI DOI 10.48550/ARXIV.1707.02937

[2] Phototoxicity revisited [J].

不详 .

NATURE METHODS, 2018, 15 (10) :751-751

[3]

[Anonymous], 2017, ICLR

[4] High-resolution restoration of 3D structures from widefield images with extreme low signal-to-noise-ratio [J].

Arigovindan, Muthuvel ;

Fung, Jennifer C. ;

Elnatan, Daniel ;

Mennella, Vito ;

Chan, Yee-Hung Mark ;

Pollard, Michael ;

Branlund, Eric ;

Sedat, John W. ;

Agard, David A. .

PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA, 2013, 110 (43) :17344-17349

[5] On the use of deep learning for computational imaging [J].

Barbastathis, George ;

Ozcan, Aydogan ;

Situ, Guohai .

OPTICA, 2019, 6 (08) :921-943

[6]

Batson J., 2019, P 36 INT C MACH LEAR, V97, P524

[7] Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction [J].

Belthangady, Chinmay ;

Royer, Loic A. .

NATURE METHODS, 2019, 16 (12) :1215-1225

[8]

Buchholz TO, 2019, I S BIOMED IMAGING, P502, DOI [10.1109/ISBI.2019.8759519, 10.1109/isbi.2019.8759519]

[9] Content-aware image restoration for electron microscopy [J].

Buchholz, Tim-Oliver ;

Krull, Alexander ;

Shahidi, Reza ;

Pigino, Gaia ;

Jekely, Gaspar ;

Jug, Florian .

THREE-DIMENSIONAL ELECTRON MICROSCOPY, 2019, 152 :277-289

[10] Fast live simultaneous multiwavelength four-dimensional optical microscopy [J].

Carlton, Peter M. ;

Boulanger, Jerome ;

Kervrann, Charles ;

Sibarita, Jean-Baptiste ;

Salamero, Jean ;

Gordon-Messer, Susannah ;

Bressan, Debra ;

Haber, James E. ;

Haase, Sebastian ;

Shao, Lin ;

Winoto, Lukman ;

Matsuda, Atsushi ;

Kner, Peter ;

Uzawa, Satoru ;

Gustafsson, Mats ;

Kam, Zvi ;

Agard, David A. ;

Sedat, John W. .

PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA, 2010, 107 (37) :16016-16022

← 1 2 3 4 5 6 →