Revisiting I/O Behavior in Large-Scale Storage Systems: The Expected and the Unexpected

被引:32
作者
Patel, Tirthak [1 ]
Byna, Surendra [2 ]
Lockwood, Glenn K. [2 ]
Tiwari, Devesh [1 ]
机构
[1] Northeastern Univ, Boston, MA 02115 USA
[2] Lawrence Berkeley Natl Lab, Berkeley, CA USA
来源
PROCEEDINGS OF SC19: THE INTERNATIONAL CONFERENCE FOR HIGH PERFORMANCE COMPUTING, NETWORKING, STORAGE AND ANALYSIS | 2019年
基金
美国国家科学基金会;
关键词
D O I
10.1145/3295500.3356183
中图分类号
TP301 [理论、方法];
学科分类号
081202 ;
摘要
Large-scale applications typically spend a large fraction of their execution time performing I/O to a parallel storage system. However, with rapid progress in compute and storage system stack of large-scale systems, it is critical to investigate and update our understanding of the I/O behavior of large-scale applications. Toward that end, in this work, we monitor, collect and analyze a year worth of storage system data from a large-scale production parallel storage system. We perform temporal, spatial and correlative analysis of the system and uncover surprising patterns which defy existing assumptions and have important implications for future systems.
引用
收藏
页数:13
相关论文
共 59 条
[1]   The Lightweight Distributed Metric Service: A Scalable Infrastructure for Continuous Monitoring of Large Scale Computing Systems and Applications [J].
Agelastos, Anthony ;
Allan, Benjamin ;
Brandt, Jim ;
Cassella, Paul ;
Enos, Jeremy ;
Fullop, Joshi ;
Gentile, Ann ;
Monk, Steve ;
Naksinehaboon, Nichamon ;
Ogden, Jeff ;
Rajan, Mahesh ;
Showerman, Michael ;
Stevenson, Joel ;
Taerat, Narate ;
Tucker, Tom .
SC14: INTERNATIONAL CONFERENCE FOR HIGH PERFORMANCE COMPUTING, NETWORKING, STORAGE AND ANALYSIS, 2014, :154-165
[2]  
Ali N, 2009, 2009 IEEE INTERNATIONAL CONFERENCE ON CLUSTER COMPUTING AND WORKSHOPS, P86
[3]  
Amvrosiadis G., 2019, DATA STORAGE RES VIS
[4]  
[Anonymous], 2017, NERSC 2017 ANN REPOR
[5]  
[Anonymous], 2014, FAST, DOI DOI 10.1186/S12876-014-0213-Y
[6]  
[Anonymous], 2007, P ACM IEEE C SUP
[7]  
[Anonymous], 2011, P P 2011 INT C HIGH
[8]  
[Anonymous], 2013, P CRAY US GROUP C
[9]  
[Anonymous], 2017, FUTURE GENERATION CO
[10]  
[Anonymous], 2016, Proceedings of the 25th ACM International Symposium on High-Performance Parallel and Distributed Computing, HPDC '16