Resource Bundles: Using Aggregation for Statistical Large-Scale Resource Discovery and Management

被引:8
作者
Cardosa, Michael [1 ]
Chandra, Abhishek [1 ]
机构
[1] Univ Minnesota, Dept Comp Sci & Engn, Minneapolis, MN 55455 USA
基金
美国国家科学基金会;
关键词
Resource discovery; aggregation; resource management; machine learning;
D O I
10.1109/TPDS.2009.143
中图分类号
TP301 [理论、方法];
学科分类号
081202 ;
摘要
Resource discovery is an important process for finding suitable nodes that satisfy application requirements in large loosely coupled distributed systems. Besides internode heterogeneity, many of these systems also show a high degree of intranode dynamism, so that selecting nodes based only on their recently observed resource capacities can lead to poor deployment decisions resulting in application failures or migration overheads. However, most existing resource discovery mechanisms rely mainly on recent observations to achieve scalability in large systems. In this paper, we propose the notion of a resource bundle-a representative resource usage distribution for a group of nodes with similar resource usage patterns-that employs two complementary techniques to overcome the limitations of existing techniques: resource usage histograms to provide statistical guarantees for resource capacities and clustering-based resource aggregation to achieve scalability. Using trace-driven simulations and data analysis of a month-long PlanetLab trace, we show that resource bundles are able to provide high accuracy for statistical resource discovery, while achieving high scalability. We also show that resource bundles are ideally suited for identifying group-level characteristics (e. g., hot spots, total group capacity). To automatically parameterize the bundling algorithm, we present an adaptive algorithm that can detect online fluctuations in resource heterogeneity.
引用
收藏
页码:1089 / 1102
页数:14
相关论文
共 28 条
[1]  
[Anonymous], P USENIX S NETW SYST
[2]  
[Anonymous], P IEEE ACM INT WORKS
[3]  
CAPPOS J, 2008, P USENIX S NETW SYST
[4]  
CHAWATHE Y, 2003, P ACM SIGCOMM AUG
[5]   PlanetLab: An overlay testbed for broad-coverage services [J].
Chun, B ;
Culler, D ;
Roscoe, T ;
Bavier, A ;
Peterson, L ;
Wawrzoniak, M ;
Bowman, M .
ACM SIGCOMM COMPUTER COMMUNICATION REVIEW, 2003, 33 (03) :3-12
[6]  
CHUN B, 2004, P WORKSH REAL LARG D
[7]  
COHEN B, 2003, P 1 WORKSH EC P2P SY
[8]  
Foster I., 2004, The Grid2: Blueprint for a New Computing Infrastructure
[9]  
GUHA S, 2006, P ITN WORKSH PEER TO
[10]  
GUPTA A, 2005, P INT S APPL INT SAI