Workflow Management for Real-time Analysis of Lightsource Experiments

被引：22

作者：

Deslippe, Jack ^{[1
]}

Essiari, Abdelilah ^{[1
]}

Patton, Simon J. ^{[1
]}

Samak, Taghrid ^{[1
]}

Tull, Craig E. ^{[1
]}

Hexemer, Alexander ^{[1
]}

Kumar, Dinesh ^{[1
]}

Parkinson, Dilworth ^{[1
]}

Stewart, Polite ^{[1
]}

机构：

[1] Lawrence Berkeley Natl Lab, Berkeley, CA USA

来源：

2014 9TH WORKSHOP ON WORKFLOWS IN SUPPORT OF LARGE-SCALE SCIENCE (WORKS) | 2014年

关键词：

D O I：

10.1109/WORKS.2014.9

中图分类号：

TP301 [理论、方法];

学科分类号：

081202 ;

摘要：

The Advanced lightsource (ALS) is a X-ray synchrotron facility at Lawrence Berkeley National Laboratory. The ALS generates terabytes of raw and derived data each day and serves 1,000's of researchers each year. Only a subset of the data is analyzed due to barriers in terms of processing that small science teams are ill-equipped to surmount. In this paper, we discuss the development and application of a computational framework, termed SPOT, fed with synchrotron data, powered by storage, networking and compute resources at NERSC and ESnet. We describe issues and recommendations for an end-toend analysis workflow for ALS data. After one year of operation, the collection contains over 90,000 datasets (550 TB) from 85 users across three beamlines. For 16 months, beamline data taken has been promptly and automatically analyzed and annotated with metadata, allowing users to focus on analysis, conclusions and experiments.

引用

页码：31 / 40

页数：10

共 9 条

[1]

Adesanya A., 2003, CORR

[2]

[Anonymous], 2013, 4 SC WORKSH PET BIG

[3]

Barabash R., 2014, Strain and Dislocation Gradients From Diffraction: Spatially- Resolved Local Structure and Defects

[4]

Cholia S, 2010, GAT COMP ENV WORKSH, P1, DOI [10.1109/GCE.2010.5676125, DOI 10.1109/GCE.2010.5676125]

[5]

Chourou S., 2013, J APPL CRYSTALLOGRAP, V46

[6]

Dede E., 2013, P 4 INT WORKSH SCI C

[7]

MARTZ HE, 2009, XRAY IMAGING

[8]

Sarje Abhinav, 2012, HIGH PERF COMP NETW

[9]

The HDF Group, 1997, HIER DAT FORM VERS

← 1 →