NoaSci: A Numerical Object Array Library for I/O of Scientific Applications on Object Storage

被引：0

作者：

Chien, Steven W. D. ^{[1
]}

Podobas, Artur ^{[1
]}

Svedin, Martin ^{[1
]}

Tkachuk, Andriy ^{[2
]}

El Sayed, Salem ^{[3
]}

Herman, Pawel ^{[1
]}

Umanesan, Ganesan ^{[2
]}

Narasimhamurthy, Sai ^{[2
]}

Markidis, Stefano ^{[1
]}

机构：

[1] KTH Royal Inst Technol, Stockholm, Sweden

[2] Seagate Syst UK, Portsmouth, Hants, England

[3] Forschungszentrum Julich, Julich Supercomp Ctr, Julich, Germany

来源：

30TH EUROMICRO INTERNATIONAL CONFERENCE ON PARALLEL, DISTRIBUTED AND NETWORK-BASED PROCESSING (PDP 2022) | 2022年

基金：

瑞典研究理事会; 欧盟地平线“2020”;

关键词：

Object-stores; Parallel I/O for Object Stores;

D O I：

10.1109/PDP55904.2022.00034

中图分类号：

TP3 [计算技术、计算机技术];

学科分类号：

0812 ;

摘要：

The strong consistency and stateful workflow are seen as the major factors for limiting parallel I/O performance because of the need for locking and state management. While the POSIX-based I/O model dominates modern HPC storage infrastructure, emerging object storage technology can potentially improve I/O performance by eliminating these bottlenecks. Despite a wide deployment on the cloud, its adoption in HPC remains low. We argue one reason is the lack of a suitable programming interface for parallel I/O in scientific applications. In this work, we introduce NoaSci, a Numerical Object Array library for scientific applications. NoaSci supports different data formats (e.g. HDF5, binary), and focuses on supporting node-local burst buffers and object stores. We demonstrate for the first time how scientific applications can perform parallel I/O on Seagate's Motr object store through NoaSci. We evaluate NoaSci's preliminary performance using the iPIC3D space weather application and position against existing I/O methods.

引用

页码：172 / 176

页数：5