EdgeML: An AutoML Framework for Real-Time Deep Learning on the Edge

被引:33
作者
Zhao, Zhihe [1 ]
Wang, Kai [2 ]
Ling, Neiwen [1 ]
Xing, Guoliang [1 ]
机构
[1] Chinese Univ Hong Kong, Hong Kong, Peoples R China
[2] Duke Univ, Durham, NC USA
来源
PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON INTERNET-OF-THINGS DESIGN AND IMPLEMENTATION, IOTDI 2021 | 2021年
关键词
Reinforcement Learning; Edge Computing; Deep Neural Network; INTERNET;
D O I
10.1145/3450268.3453520
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
In recent years, deep learning algorithms are increasingly adopted by a wide range of data-intensive and time-critical Internet of Things (IoT) applications. As a result, several new approaches, including model partition/offloading and progressive neural architecture, have been proposed to address the challenge of deploying the computation-intensive deep neural network (DNN) models on resource-constrained edge devices. However, the performance of existing approaches is highly affected by runtime dynamics. For example, offloading workload from edge to cloud suffers from communication delays and the efficiency of progressive neural architecture supporting early-exit DNN executions relies on input characteristics. In this paper, we introduce EdgeML, an AutoML framework that provides flexible and fine-grained DNN model execution control by combining workload offloading mechanism and dynamic progressive neural architecture. To achieve desirable latency-accuracy-energy system performance on edge platforms, EdgeML adopts reinforcement learning to automatically update model execution policy in response to runtime dynamics in real-time. We implement EdgeML for several widely used DNN models on the latest edge devices. Comparing to existing approaches, our experiments show that EdgeML achieves up to 8x performance improvement under dynamic environments.
引用
收藏
页码:133 / 144
页数:12
相关论文
共 41 条
[11]  
Han S, 2016, Arxiv, DOI arXiv:1510.00149
[12]   Deep Residual Learning for Image Recognition [J].
He, Kaiming ;
Zhang, Xiangyu ;
Ren, Shaoqing ;
Sun, Jian .
2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2016, :770-778
[13]   AMC: AutoML for Model Compression and Acceleration on Mobile Devices [J].
He, Yihui ;
Lin, Ji ;
Liu, Zhijian ;
Wang, Hanrui ;
Li, Li-Jia ;
Han, Song .
COMPUTER VISION - ECCV 2018, PT VII, 2018, 11211 :815-832
[14]  
Hinton G, 2015, Arxiv, DOI arXiv:1503.02531
[15]   In-Datacenter Performance Analysis of a Tensor Processing Unit [J].
Jouppi, Norman P. ;
Young, Cliff ;
Patil, Nishant ;
Patterson, David ;
Agrawal, Gaurav ;
Bajwa, Raminder ;
Bates, Sarah ;
Bhatia, Suresh ;
Boden, Nan ;
Borchers, Al ;
Boyle, Rick ;
Cantin, Pierre-luc ;
Chao, Clifford ;
Clark, Chris ;
Coriell, Jeremy ;
Daley, Mike ;
Dau, Matt ;
Dean, Jeffrey ;
Gelb, Ben ;
Ghaemmaghami, Tara Vazir ;
Gottipati, Rajendra ;
Gulland, William ;
Hagmann, Robert ;
Ho, C. Richard ;
Hogberg, Doug ;
Hu, John ;
Hundt, Robert ;
Hurt, Dan ;
Ibarz, Julian ;
Jaffey, Aaron ;
Jaworski, Alek ;
Kaplan, Alexander ;
Khaitan, Harshit ;
Killebrew, Daniel ;
Koch, Andy ;
Kumar, Naveen ;
Lacy, Steve ;
Laudon, James ;
Law, James ;
Le, Diemthu ;
Leary, Chris ;
Liu, Zhuyuan ;
Lucke, Kyle ;
Lundin, Alan ;
MacKean, Gordon ;
Maggiore, Adriana ;
Mahony, Maire ;
Miller, Kieran ;
Nagarajan, Rahul ;
Narayanaswami, Ravi .
44TH ANNUAL INTERNATIONAL SYMPOSIUM ON COMPUTER ARCHITECTURE (ISCA 2017), 2017, :1-12
[16]   Neurosurgeon: Collaborative Intelligence Between the Cloud and Mobile Edge [J].
Kang, Yiping ;
Hauswald, Johann ;
Gao, Cao ;
Rovinski, Austin ;
Mudge, Trevor ;
Mars, Jason ;
Tang, Lingjia .
ACM SIGPLAN NOTICES, 2017, 52 (04) :615-629
[17]  
Kloss Carey, 2017, Intel nervanaT neural network processor: Architecture update
[18]  
Krizhevsky Alex, 2009, Handbook of Systemic Autoimmune Diseases
[19]  
Laskaridis S., 2020, P 26 ANN INT C MOB C, P1
[20]   Fast Algorithms for Convolutional Neural Networks [J].
Lavin, Andrew ;
Gray, Scott .
2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2016, :4013-4021