NoaSci: A Numerical Object Array Library for I/O of Scientific Applications on Object Storage

被引:0
|
作者
Chien, Steven W. D. [1 ]
Podobas, Artur [1 ]
Svedin, Martin [1 ]
Tkachuk, Andriy [2 ]
El Sayed, Salem [3 ]
Herman, Pawel [1 ]
Umanesan, Ganesan [2 ]
Narasimhamurthy, Sai [2 ]
Markidis, Stefano [1 ]
机构
[1] KTH Royal Inst Technol, Stockholm, Sweden
[2] Seagate Syst UK, Portsmouth, Hants, England
[3] Forschungszentrum Julich, Julich Supercomp Ctr, Julich, Germany
来源
30TH EUROMICRO INTERNATIONAL CONFERENCE ON PARALLEL, DISTRIBUTED AND NETWORK-BASED PROCESSING (PDP 2022) | 2022年
基金
瑞典研究理事会; 欧盟地平线“2020”;
关键词
Object-stores; Parallel I/O for Object Stores;
D O I
10.1109/PDP55904.2022.00034
中图分类号
TP3 [计算技术、计算机技术];
学科分类号
0812 ;
摘要
The strong consistency and stateful workflow are seen as the major factors for limiting parallel I/O performance because of the need for locking and state management. While the POSIX-based I/O model dominates modern HPC storage infrastructure, emerging object storage technology can potentially improve I/O performance by eliminating these bottlenecks. Despite a wide deployment on the cloud, its adoption in HPC remains low. We argue one reason is the lack of a suitable programming interface for parallel I/O in scientific applications. In this work, we introduce NoaSci, a Numerical Object Array library for scientific applications. NoaSci supports different data formats (e.g. HDF5, binary), and focuses on supporting node-local burst buffers and object stores. We demonstrate for the first time how scientific applications can perform parallel I/O on Seagate's Motr object store through NoaSci. We evaluate NoaSci's preliminary performance using the iPIC3D space weather application and position against existing I/O methods.
引用
收藏
页码:172 / 176
页数:5
相关论文
共 50 条
  • [21] Using Centralized I/O Scheduling Service(CISS) to Improve Cloud Object Storage Performance
    Shi, Xiao
    Hu, Detian
    Tang, Hongwei
    Zheng, Xiaohui
    Zhao, Xiaofang
    2018 IEEE INT CONF ON PARALLEL & DISTRIBUTED PROCESSING WITH APPLICATIONS, UBIQUITOUS COMPUTING & COMMUNICATIONS, BIG DATA & CLOUD COMPUTING, SOCIAL COMPUTING & NETWORKING, SUSTAINABLE COMPUTING & COMMUNICATIONS, 2018, : 361 - 368
  • [22] An object-oriented interface for the dynamic memory management of sparse discrete mathematical operators in numerical scientific applications
    Manzini, G
    Mazet, S
    SOFTWARE-PRACTICE & EXPERIENCE, 2002, 32 (07): : 621 - 644
  • [23] Lightweight I/O for scientific applications
    Oldfield, Ron A.
    Ward, Lee
    Riesen, Rolf
    Maccabe, Arthur B.
    Widener, Patrick
    Kordenbrock, Todd
    2006 IEEE INTERNATIONAL CONFERENCE ON CLUSTER COMPUTING, VOLS 1 AND 2, 2006, : 152 - +
  • [24] Managing Object Identity with less I/O operations
    de Almeida, Fabio Renato
    Valencio, Carlos Roberto
    Ferrizzi, Andre Cid
    Tronco, Mario Luiz
    Gratao de Souza, Rogeria Cristiane
    2012 13TH INTERNATIONAL CONFERENCE ON PARALLEL AND DISTRIBUTED COMPUTING, APPLICATIONS, AND TECHNOLOGIES (PDCAT 2012), 2012, : 213 - 218
  • [25] Applying Fortran 90 and object-oriented techniques to scientific applications
    Norton, CD
    Decyk, V
    Slottow, J
    OBJECT-ORIENTED TECHNOLOGY: ECOOP'98 WORKSHOP READER, 1998, 1543 : 462 - 463
  • [26] An object-oriented architecture for applications of scientific visualization and mathematical modeling
    Semenov, VA
    Krylov, PB
    Morozov, SV
    Tarlapan, OA
    PROGRAMMING AND COMPUTER SOFTWARE, 2000, 26 (02) : 74 - 83
  • [27] An object-oriented architecture for applications of scientific visualization and mathematical modeling
    V. A. Semenov
    P. B. Krylov
    S. V. Morozov
    O. A. Tarlapan
    Programming and Computer Software, 2000, 26 : 74 - 83
  • [28] MOSIQS: Persistent Memory Object Storage With Metadata Indexing and Querying for Scientific Computing
    Khan, Awais
    Sim, Hyogi
    Vazhkudai, Sudharshan S.
    Kim, Youngjae
    IEEE ACCESS, 2021, 9 : 85217 - 85231
  • [29] Object-process based graphics recognition class library: Principles and applications
    Wenyin, Liu
    Dori, Dov
    Software - Practice and Experience, 1999, 29 (15): : 1355 - 1378
  • [30] An object orientated library for evolution programs - with applications for partitioning of finite element meshes
    Zola, J
    Lacinski, L
    Wyrzykowski, R
    IUTAM SYMPOSIUM ON EVOLUTIONARY METHODS IN MECHANICS, 2004, 117 : 351 - 360