State-of-the-art Survey of Scheduling and Resource Management Technology for Colocation Jobs

被引：0
|
作者：
Wang K.-J. [1 ]
Jia T. [3 ]
Li Y. [1 ,2 ]
机构：
[1] School of Software and Microelectronics, Peking University, Beijing
[2] National Engineering Research Center for Software Engineering, Peking University, Beijing
[3] School of Electronics and Computer Science, Peking University, Beijing
来源：
Ruan Jian Xue Bao/Journal of Software | 2020年 / 31卷 / 10期
关键词：
Internet datacenter; Job scheduling; Performance interference; Resource management technology; Resource utilization;
D O I：
10.13328/j.cnki.jos.006066
中图分类号：
C93 [管理学];
学科分类号：
12 ; 1201 ; 1202 ; 120202 ;
摘要：
Data center is not only an important IT infrastructure, but also a key support for enterprise Internet application. However, the resource utilization of data center is pretty low (only 10%~20%), which leads to a large amount of waste of resources, brings a huge extra operation and maintenance cost, and becomes a key problem restricting enterprises to improve the computing efficiency. By colocating online services and offline tasks, colocation can effectively improve the resource utilization rate of data center, which has become a research hotspot in academia and industry. This paper analyzes the characteristics of online services and offline tasks, and discusses the technical challenges faced by the performance interference between services and jobs. This paper summarizes the key technologies from the aspects of performance interference model, job scheduling, resource isolation and dynamic resource allocation, and discusses the application and effect of colocation systems in the industry with four typical colocation system. At the end of this paper, the future research direction is presented. © Copyright 2020, Institute of Software, the Chinese Academy of Sciences. All rights reserved.
引用
收藏
页码：3100 / 3119
页数：19
相关论文
共 96 条
  • [1] Arman S, Et al., United States data center energy usage report, (2016)
  • [2] Dean J, Ghemawat S., MapReduce: Simplified data processing on large clusters, Communications of the ACM, 51, 1, (2008)
  • [3] Zaharia M, Chowdhury M, Franklin MJ, Et al., Spark: Cluster computing with working sets, Proc. of the Usenix Conf. on Hot Topics in Cloud Computing, (2010)
  • [4] Vavilapalli VK, Murthy AC, Douglas C, Et al., Apache hadoop yarn: Yet another resource negotiator, Proc. of the 4th Annual Symp. on Cloud Computing, pp. 1-16, (2013)
  • [5] Hindman B, Konwinski A, Zaharia M, Et al., Mesos: A platform for fine-grained resource sharing in the data center, NSDI, 11, 2011, pp. 22-22, (2011)
  • [6] Burns B, Grant B, Oppenheimer D, Et al., Borg, Omega, and Kubernetes, Queue, 14, 1, pp. 70-93, (2016)
  • [7] Du XY, Lu W, Zhang F., History, present, and future of big data management systems, Ruan Jian Xue Bao/Journal of Software, 30, 1, pp. 127-141, (2019)
  • [8] Baidu large-scale strategic colocation system evolution
  • [9] Verma A, Pedrosa L, Korupolu M, Et al., Large-scale cluster management at Google with Borg, Proc. of the 10th European Conf. on Computer Systems, pp. 1-17, (2015)
  • [10] Chen S, Delimitrou C, Martinez JF., PARTIES: QoS-aware resource partitioning for multiple interactive services, (2019)
12345678910