Dynamic Rate Control for Topic-based Pub/Sub Systems

被引:1
作者
Tsenos, Michalis [1 ]
Kalogeraki, Vana [2 ]
机构
[1] Athens Univ Econ & Business, Dept Infomat, Athens, Greece
[2] Athens Univ Econ & Business, Athens, Greece
来源
2021 22ND IEEE INTERNATIONAL CONFERENCE ON MOBILE DATA MANAGEMENT (MDM 2021) | 2021年
基金
欧盟地平线“2020”;
关键词
Apache Kafka; Rate Control; Distributed systems;
D O I
10.1109/MDM52706.2021.00057
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Pub/sub systems have been widely utilized in the industry as the connecting piece between high rate producers or mobile clients and end-services, due to their ability to handle messages of high volume and velocity and achieve high throughput. Apache Kafka is one of the most popular Big Data messaging systems. Although Kafka's modular Consumer API allows businesses to take advantage of the rich set of features it provides, it often suffers from improper configured topics/partitions or load imbalances due to spikes or high volume of messages injected in the same topic. In our research our goal is to develop a novel framework which investigates smart queuing mechanisms to overcome these issues and deal effectively with sudden bursts and overloads which are frequently experienced in these systems. Our approach provides rate control capabilities to the Kafka's consumer API which allows us to effectively meet the requirements of different end-services without interference among them.
引用
收藏
页码:272 / 273
页数:2
相关论文
共 50 条
[41]   Rate Control for Screen Content Coding Based on Picture Classification [J].
Guo, Yaoyao ;
Li, Bin ;
Sun, Songlin ;
Xu, Jizheng .
2015 VISUAL COMMUNICATIONS AND IMAGE PROCESSING (VCIP), 2015,
[42]   LAPLACE DISTRIBUTION BASED CTU LEVEL RATE CONTROL FOR HEVC [J].
Si, Junjun ;
Ma, Siwei ;
Wang, Shiqi ;
Gao, Wen .
2013 IEEE INTERNATIONAL CONFERENCE ON VISUAL COMMUNICATIONS AND IMAGE PROCESSING (IEEE VCIP 2013), 2013,
[43]   Rate control algorithm based on frame complexity estimation for MVC [J].
Yan, Tao ;
An, Ping ;
Shen, Liquan ;
Zhang, Zhaoyang .
VISUAL COMMUNICATIONS AND IMAGE PROCESSING 2010, 2010, 7744
[44]   SSIM-Based Perceptual Rate Control for Video Coding [J].
Ou, Tao-Sheng ;
Huang, Yi-Hsin ;
Chen, Homer H. .
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2011, 21 (05) :682-691
[45]   Rate-Control and Queueing of Backhaul Downstream Traffic for Mobile Wireless Systems [J].
Choi, Young-June ;
Kim, Kyungtae ;
Shin, Kang G. .
WIRELESS PERSONAL COMMUNICATIONS, 2011, 60 (01) :125-143
[46]   Energy-efficient control of rate and power in DS-CDMA systems [J].
Berggren, F ;
Kim, SL .
IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, 2004, 3 (03) :725-733
[47]   RATE CONTROL FOR PSD LIMITED MULTIPLE ACCESS SYSTEMS THROUGH LINEAR PROGRAMMING [J].
Leshem, Amir ;
Zehavi, Ephraim .
2011 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, 2011, :3220-3223
[48]   Rate-Control and Queueing of Backhaul Downstream Traffic for Mobile Wireless Systems [J].
Young-June Choi ;
Kyungtae Kim ;
Kang G. Shin .
Wireless Personal Communications, 2011, 60 :125-143
[49]   Global Rate-Distortion Optimization-Based Rate Control for HEVC HDR Coding [J].
Zhou, Mingliang ;
Wei, Xuekai ;
Wang, Shiqi ;
Kwong, Sam ;
Fong, Chi-Keung ;
Wong, Peter H. W. ;
Yuen, Wilson Y. F. .
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2020, 30 (12) :4648-4662
[50]   Cauchy based rate-distortion optimization model for H.264 rate control [J].
Eiamjumrus, N. ;
Aramvith, S. .
2006 IEEE ASIA PACIFIC CONFERENCE ON CIRCUITS AND SYSTEMS, 2006, :77-+