Amber: A Debuggable Dataflow System Based on the Actor Model

被引:14
|
作者
Kumar, Avinash [1 ]
Wang, Zuozhi [1 ]
Ni, Shengquan [1 ]
Li, Chen [1 ]
机构
[1] UC Irvine, Dept Comp Sci, Irvine, CA 92697 USA
来源
PROCEEDINGS OF THE VLDB ENDOWMENT | 2020年 / 13卷 / 05期
关键词
D O I
10.14778/3377369.3377381
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
A long-running analytic task on big data often leaves a developer in the dark without providing valuable feedback about the status of the execution. In addition, a failed job that needs to restart from scratch can waste earlier computing resources. An effective method to address these issues is to allow the developer to debug the task during its execution, which is unfortunately not supported by existing big data solutions. In this paper we develop a system called Amber that supports responsive debugging during the execution of a workflow task. After starting the execution, the developer can pause the job at will, investigate the states of the cluster, modify the job, and resume the computation. She can also set conditional breakpoints to pause the execution when certain conditions are satisfied. In this way, the developer can gain a much better understanding of the run-time behavior of the execution and more easily identify issues in the job or data. Amber is based on the actor model, a distributed computing paradigm that provides concurrent units of computation using actors. We give a full specification of Amber, and implement it on top of the Orleans system. Our experiments show its high performance and usability of debugging on computing clusters.
引用
收藏
页码:740 / 753
页数:14
相关论文
共 50 条
  • [1] Port Based Actor Model with Kahn Process Network Model and Decidable Dataflow Model
    Oh, Hyunok
    JOURNAL OF SIGNAL PROCESSING SYSTEMS FOR SIGNAL IMAGE AND VIDEO TECHNOLOGY, 2013, 71 (01): : 75 - 88
  • [2] Port Based Actor Model with Kahn Process Network Model and Decidable Dataflow Model
    Hyunok Oh
    Journal of Signal Processing Systems, 2013, 71 : 75 - 88
  • [3] Actor-Based Parallel Dataflow Analysis
    Rodriguez, Jonathan
    Lhotak, Ondrej
    COMPILER CONSTRUCTION, 2011, 6601 : 179 - 197
  • [4] A Multicore DSP HEVC Decoder Using an Actor-based Dataflow Model
    Chavarrias, M.
    Pescador, F.
    Garrido, M. J.
    Juarez, E.
    Sanz, C.
    2015 IEEE INTERNATIONAL CONFERENCE ON CONSUMER ELECTRONICS (ICCE), 2015, : 370 - 371
  • [5] A DSP-Based HEVC Decoder Implementation Using an Actor Language Dataflow Model
    Chavarrias, M.
    Pescador, F.
    Garrido, M. J.
    Juarez, E.
    Raulet, M.
    IEEE TRANSACTIONS ON CONSUMER ELECTRONICS, 2013, 59 (04) : 839 - 847
  • [6] A Multicore DSP HEVC Decoder Using an Actor-based Dataflow Model and OpenMP
    Chavarrias, M.
    Pescador, F.
    Garrido, M. J.
    Juarez, E.
    Sanz, C.
    IEEE TRANSACTIONS ON CONSUMER ELECTRONICS, 2015, 61 (02) : 236 - 244
  • [7] Actor Merging for Dataflow Process Networks
    Boutellier, Jani
    Ersfolk, Johan
    Lilius, Johan
    Mattavelli, Marco
    Roquier, Ghislain
    Silven, Olli
    IEEE TRANSACTIONS ON SIGNAL PROCESSING, 2015, 63 (10) : 2496 - 2508
  • [8] A Timely Dataflow Model and the Naiad System
    Abadi, Martin
    ELECTRONIC PROCEEDINGS IN THEORETICAL COMPUTER SCIENCE, 2013, (120): : 1 - 1
  • [9] Parallel computations based on the dataflow model
    Veremeev, A.P.
    Izvestiya Akademii Nauk. Teoriya i Sistemy Upravleniya, 1993, (03): : 212 - 225