High performance data management and analysis for tomography

被引:10
作者
Blair, Justin [1 ]
Canon, Richard S. [1 ]
Deslippe, Jack [1 ]
Essiari, Abdelilah [1 ]
Hexemer, Alexander [1 ]
Mac Dowell, Alastair A. [1 ]
Parkinson, Dilworth Y. [1 ]
Patton, Simon J. [1 ]
Ramakrishnan, Lavanya [1 ]
Tamura, Nobumichi [1 ]
Tierney, Brian L. [1 ]
Tull, Craig E. [1 ]
机构
[1] Univ Calif Berkeley, Lawrence Berkeley Natl Lab, Berkeley, CA 94720 USA
来源
DEVELOPMENTS IN X-RAY TOMOGRAPHY IX | 2014年 / 9212卷
关键词
MicroCT; Tomography; big data; supercomputer; high performance computing; synchrotron; light source; workflow;
D O I
10.1117/12.2069862
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
The Advanced Light Source (ALS) is a third-generation synchrotron X-ray source that operates as a user facility with more than 40 beamlines hosting over 2000 users per year from around the world. Users of the Hard X-ray Micro-Tomography Beamline (8.3.2) often collect more than 1 Terabyte of raw data per day that in turn generates additional Terabytes of processed data. The data rate continues to increase rapidly due to faster detectors and new sample automation capabilities. We will present the development and deployment of a computational pipeline, fed by data from the ALS, and powered by the storage, networking, and computing resources of the local National Energy Research Scientific Computing Center (NERSC) and the Energy Sciences Network (ESNET). After one year of operation, the system contained 70,000 datasets and 350 TB of data from 85 users. All datasets now collected at the Hard X-ray Tomography Beamline are automatically reconstructed using parameters set by users and/or that are automatically detected from the data acquisition control system. Results are presented to users for visualization through a secure web portal. Users can then download their data or launch a (currently limited but) growing number of operations based on the data-such as filtering, segmentation, and simulation. The massive computational resources of NERSC are thus made available on a level that is easily accessible to the full range of micro-tomography users.
引用
收藏
页数:9
相关论文
共 4 条
[1]  
[Anonymous], 1997, HIER DAT FORM VERS 5
[2]  
[Anonymous], 2013, 4 SC WORKSH PET BIG
[3]  
Cholia S, 2010, GAT COMP ENV WORKSH, P1, DOI [10.1109/GCE.2010.5676125, DOI 10.1109/GCE.2010.5676125]
[4]  
MARTZ HE, 2009, XRAY IMAGING