Imaging SKA-scale data in three different computing environments

被引：16

作者：

Dodson, R. ^{[1
]}

Vinsen, K. ^{[1
]}

Wu, C. ^{[1
]}

Popping, A. ^{[1
]}

Meyer, M. ^{[1
]}

Wicenec, A. ^{[1
]}

Quinn, P. ^{[1
]}

van Gorkom, J. ^{[2
]}

Momjian, E. ^{[3
]}

机构：

[1] Univ Western Australia, ICRAR, M468,35 Stirling Highway, Perth, WA 6009, Australia

[2] Columbia Univ, Dept Astron, Mail Code 5246,550 West 120th St, New York, NY 10027 USA

[3] Natl Radio Astron Observ, 1003 Lopezville Rd,POB O, Socorro, NM 87801 USA

来源：

ASTRONOMY AND COMPUTING | 2016年 / 14卷

基金：

美国国家科学基金会;

关键词：

Methods: data analysis; Parallel architectures: multicore architectures; Distributed architectures: cloud computing; CHILES; LARGE ARRAY;

D O I：

10.1016/j.ascom.2015.10.007

中图分类号：

P1 [天文学];

学科分类号：

0704 ;

摘要：

We present the results of our investigations into options for the computing platform for the imaging pipeline in the CHILES project, an ultra-deep HI pathfinder for the era of the Square Kilometre Array. CHILES pushes the current computing infrastructure to its limits and understanding how to deliver the images from this project is clarifying the Science Data Processing requirements for the SKA. We have tested three platforms: a moderately sized cluster, a massive High Performance Computing (HPC) system, and the Amazon Web Services (AWS) cloud computing platform. We have used well-established tools for data reduction and performance measurement to investigate the behaviour of these platforms for the complicated access patterns of real-life Radio Astronomy data reduction. All of these platforms have strengths and weaknesses and the system tools allow us to identify and evaluate them in a quantitative manner. With the insights from these tests we are able to complete the imaging pipeline processing on both the HPC platform and also on the cloud computing platform, which paves the way for meeting big data challenges in the era of SKA in the field of Radio Astronomy. We discuss the implications that all similar projects will have to consider, in both performance and costs, to make recommendations for the planning of Radio Astronomy imaging workflows. (C) 2015 Elsevier B.V. All rights reserved.

引用

页码：8 / 22

页数：15

共 23 条

[1]

AWS, 2015, BOT PYTH INT AM WEB

[2]

Beckett G., 2015, COMMUNICATION

[3] Weighting interferometric data for direct imaging [J].

Boone, Frederic .

EXPERIMENTAL ASTRONOMY, 2013, 36 (1-2) :77-104

[4]

Bowden T, 2009, THE PROC FILE SYSTEM

[5]

Cornwell T., 2015, TECHNICAL REPORT

[6]

Dodson R., 2013, TECHNICAL REPORT

[7]

Fernandez X., 2015, AM ASTR SOC M, P42703

[8] A PILOT FOR A VERY LARGE ARRAY H I DEEP FIELD [J].

Fernandez, Ximena ;

van Gorkom, J. H. ;

Hess, Kelley M. ;

Pisano, D. J. ;

Kreckel, Kathryn ;

Momjian, Emmanuel ;

Popping, Attila ;

Oosterloo, Tom ;

Chomiuk, Laura ;

Verheijen, M. A. W. ;

Henning, Patricia A. ;

Schiminovich, David ;

Bershady, Matthew A. ;

Wilcots, Eric M. ;

Scoville, Nick .

ASTROPHYSICAL JOURNAL LETTERS, 2013, 770 (02)

[9]

Golap K., 2015, COMMUNICATION

[10]

Hogbom J. A., 1974, Astronomy and Astrophysics Supplement Series, V15, P417

← 1 2 3 →