Dynamic Rate Control for Topic-based Pub/Sub Systems

被引：1

作者：

Tsenos, Michalis ^{[1
]}

Kalogeraki, Vana ^{[2
]}

机构：

[1] Athens Univ Econ & Business, Dept Infomat, Athens, Greece

[2] Athens Univ Econ & Business, Athens, Greece

来源：

2021 22ND IEEE INTERNATIONAL CONFERENCE ON MOBILE DATA MANAGEMENT (MDM 2021) | 2021年

基金：

欧盟地平线“2020”;

关键词：

Apache Kafka; Rate Control; Distributed systems;

D O I：

10.1109/MDM52706.2021.00057

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

Pub/sub systems have been widely utilized in the industry as the connecting piece between high rate producers or mobile clients and end-services, due to their ability to handle messages of high volume and velocity and achieve high throughput. Apache Kafka is one of the most popular Big Data messaging systems. Although Kafka's modular Consumer API allows businesses to take advantage of the rich set of features it provides, it often suffers from improper configured topics/partitions or load imbalances due to spikes or high volume of messages injected in the same topic. In our research our goal is to develop a novel framework which investigates smart queuing mechanisms to overcome these issues and deal effectively with sudden bursts and overloads which are frequently experienced in these systems. Our approach provides rate control capabilities to the Kafka's consumer API which allows us to effectively meet the requirements of different end-services without interference among them.

引用

页码：272 / 273

页数：2

共 50 条

[21] QUADRATIC ρ-DOMAIN BASED RATE CONTROL ALGORITHM FOR HEVC [J].

Wang, Shanshe ;

Ma, Siwei ;

Wang, Shiqi ;

Zhao, Debin ;

Gao, Wen .

2013 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2013, :1695-1699

[22] Traffic Rate Control Based on Incentive Stackelberg Strategy [J].

Wang, Dan ;

Qian, Xiaolong ;

Jin, Xiaozheng .

PROCEEDINGS OF THE 2012 24TH CHINESE CONTROL AND DECISION CONFERENCE (CCDC), 2012, :1284-1287

[23] Adaptive rate control algorithm for low power video coding systems [J].

F. Pan ;

Z. G. Li ;

K. P. Lim ;

D. J. Wu ;

R. S. Yu .

Multidimensional Systems and Signal Processing, 2007, 18 :5-15

[24] A novel joint power and rate control algorithm in cellular radio systems [J].

Xue, Wenhu ;

Xi, Zemin ;

Gan, Zhongmin .

TENCON 2005 - 2005 IEEE REGION 10 CONFERENCE, VOLS 1-5, 2006, :2476-+

[25] Adaptive rate control algorithm for low power video coding systems [J].

Pan, F. ;

Li, Z. G. ;

Lim, K. P. ;

Wu, D. J. ;

Yu, R. S. .

MULTIDIMENSIONAL SYSTEMS AND SIGNAL PROCESSING, 2007, 18 (01) :5-15

[26] A token-bucket based rate control algorithm with maximum and minimum rate constraints [J].

Kim, Han Seok ;

Park, Eun-Chan ;

Heo, Seo Weon .

IEICE TRANSACTIONS ON COMMUNICATIONS, 2008, E91B (05) :1623-1626

[27] A novel real-time rate control algorithm based on the rate distortion theory [J].

Li Zi-yin ;

Zhu Shan-an .

Proceedings of 2005 Chinese Control and Decision Conference, Vols 1 and 2, 2005, :624-627

[28] A novel rate predictor based on quantized DCT indices and its rate control mechanism [J].

Chang, SC ;

Yang, JF ;

Lee, CF ;

Hwang, JN .

SIGNAL PROCESSING-IMAGE COMMUNICATION, 2003, 18 (06) :427-441

[29] Content-based rate control for low bit-rate video applications [J].

Shi, KW ;

Cai, A ;

Sun, JG .

VISUAL COMMUNICATIONS AND IMAGE PROCESSING 2000, PTS 1-3, 2000, 4067 :527-532

[30] Role based access control in distributed systems [J].

Vandenwauver, M ;

Govaerts, R ;

Vandewalle, J .

COMMUNICATIONS AND MULTIMEDIA SECURITY, VOL 3, 1997, :169-177

← 1 2 3 4 5 →