On the Optimality of Data Exchange for Master-Aided Edge Computing Systems

被引:3
作者
Chen, Haoning [1 ]
Long, Junfeng [1 ]
Ma, Shuai [2 ]
Tang, Mingjian [3 ]
Wu, Youlong [1 ]
机构
[1] Shanghai Tech Univ, Sch Informat Sci & Technol, Shanghai 201210, Peoples R China
[2] Peng Cheng Lab, Shenzhen 518055, Peoples R China
[3] Westpac Banking Corp, Sydney, NSW 2000, Australia
关键词
Distributed computing; MapReduce; computation; communication; FUNDAMENTAL LIMITS; PERFORMANCE;
D O I
10.1109/TCOMM.2023.3238373
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Edge computing has recently garnered significant interest in many Internet of Things (IoT) applications. However, the excessive overhead during data exchange still remains an open challenge, especially for large-scale data processing tasks. This paper considers a master-aided distributed computing system with multiple edge computing nodes and a master node, where the master node helps edge nodes compute output functions. We propose a coded scheme to reduce the communication latency by exploiting computation and communication capabilities of all nodes and creating coded multicast opportunities. More importantly, we prove that the proposed scheme is always optimal, i.e., achieving the minimum communication latency, for arbitrary computing and storage abilities at the master. This extends the previous optimality results in the extreme cases (either the master could compute all input files or compute nothing) to the general case. Finally, numerical results and TeraSort experiments demonstrate that our schemes can greatly reduce the communication latency compared with the existing schemes.
引用
收藏
页码:1364 / 1376
页数:13
相关论文
共 40 条
[1]  
Ahmad F, 2012, ASPLOS XVII: SEVENTEENTH INTERNATIONAL CONFERENCE ON ARCHITECTURAL SUPPORT FOR PROGRAMMING LANGUAGES AND OPERATING SYSTEMS, P61
[2]   A scalable, commodity data center network architecture [J].
Al-Fares, Mohammad ;
Loukissas, Alexander ;
Vahdat, Amin .
ACM SIGCOMM COMPUTER COMMUNICATION REVIEW, 2008, 38 (04) :63-74
[3]  
Dean J, 2004, USENIX ASSOCIATION PROCEEDINGS OF THE SIXTH SYMPOSIUM ON OPERATING SYSTEMS DESIGN AND IMPLEMENTATION (OSDE '04), P137
[4]  
Golub H., 1983, MATRIX COMPUTATIONS
[5]   iShuffle: Improving Hadoop Performance with Shuffle-on-Write [J].
Guo, Yanfei ;
Rao, Jia ;
Cheng, Dazhao ;
Zhou, Xiaobo .
IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, 2017, 28 (06) :1649-1662
[6]   Fundamental Limits of Caching in Wireless D2D Networks [J].
Ji, Mingyue ;
Caire, Giuseppe ;
Molisch, Andreas F. .
IEEE TRANSACTIONS ON INFORMATION THEORY, 2016, 62 (02) :849-869
[7]   Cascaded Coded Distributed Computing Schemes Based on Placement Delivery Arrays [J].
Jiang, Jing ;
Qu, Lingxiao .
IEEE ACCESS, 2020, 8 :221385-221395
[8]  
Kiamari M, 2017, CONF REC ASILOMAR C, P536, DOI 10.1109/ACSSC.2017.8335397
[9]  
Kiamari M, 2017, IEEE GLOB COMM CONF
[10]   Resolvable Designs for Speeding Up Distributed Computing [J].
Konstantinidis, Konstantinos ;
Ramamoorthy, Aditya .
IEEE-ACM TRANSACTIONS ON NETWORKING, 2020, 28 (04) :1657-1670