Flow control as a stochastic optimal control problem with incomplete information

被引：21

作者：

Miller B.M. ^{[1
]}

Avrachenkov K.E. ^{[2
]}

Stepanyan K.V. ^{[1
]}

Miller G.B. ^{[3
]}

机构：

[1] Institute for Information Transmission Problems, RAS, Moscow

[2] INRIA, Sophia-Antipolis

[3] Moscow Aviation Institute, State University of Aerospace Technologies, Moscow

来源：

Problems of Information Transmission | 2005年 / 41卷 / 2期

基金：

俄罗斯基础研究基金会;

关键词：

Optimal Control Problem; Congestion Control; Loss Process; Link State; Stochastic Optimal Control;

D O I：

10.1007/s11122-005-0020-8

中图分类号：

学科分类号：

摘要：

A nonlinear stochastic control problem related to flow control is considered. It is assumed that the state of a link is described by a controlled hidden Markov process with a finite state set, while the loss flow is described by a counting process with intensity depending on a current transmission rate and an unobserved link state. The control is the transmission rate, and it has to be chosen as a nonanticipating process depending on the observation of the loss process. The aim of the control is to achieve the maximum of some utility function that takes into account losses of the transmitted information. Originally, the problem belongs to the class of stochastic control problems with incomplete information; however, optimal filtering equations that provide estimation of the current link state based on observations of the loss process allow one to reduce the problem to a standard stochastic control problem with full observations. Then a necessary optimality condition is derived in the form of a stochastic maximum principle, which allows us to obtain explicit analytic expressions for the optimal control in some particular cases. Optimal and suboptimal controls are investigated and compared with the flow control schemes used in TCP/IP (Transmission Control Protocols/Internet Protocols) networks. In particular, the optimal control demonstrates a much smoother behavior than the TCP/IP congestion control currently used. © 2005 Pleiades Publishing, Inc.

引用

页码：150 / 170

页数：20

共 34 条

[1]

Jacobson V., Congestion Avoidance and Control, Proc. ACM SIGCOMM'88, pp. 314-329, (1988)

[2]

Allman M., Paxson V., Stevens W., TCP Congestion Control, (1999)

[3]

Ramakrishnan K., Floyd S., Black D., The Addition of Explicit Congestion Notification (ECN) to IP, (2001)

[4]

Ramakrishnan K., Jain R., A Binary Feedback Scheme for Congestion Avoidance in Computer Networks with Connectionless Network Layer, ACM Trans. Comput. Syst., 8, 2, pp. 158-181, (1990)

[5]

Altman E., Avrachenkov K., Barakat C., A Stochastic Model of TCP/IP with Stationary Random Losses, Comput. Commun. Rev., 30, 4, pp. 231-242, (2000)

[6]

Altman E., Avrachenkov K., Barakat C., TCP in Presence of Bursty Losses, Perform. Eval., 42, 2-3, pp. 129-147, (2000)

[7]

Barakat C., TCP/IP Modeling and Validation, IEEE Network, 15, 3, pp. 38-47, (2001)

[8]

Altman E., Avrachenkov K.E., Barakat C., Dube P., TCP over a Multi-state Markovian Path, Performance and QoS of Next Generation Networking, pp. 103-122, (2000)

[9]

Davis M.H.A., Markov Models and Optimization, (1993)

[10]

Elliott R.J., Aggoun L., Moore J.B., Hidden Markov Models: Estimation and Control, (1995)

← 1 2 3 4 →