Reinforcement learning based dynamic distributed routing scheme for mega LEO satellite networks

被引：26

作者：

Huang, Yixin ^{[1
]}

Wu, Shufan ^{[1
]}

Kang, Zeyu ^{[1
]}

Mu, Zhongcheng ^{[1
]}

Huang, Hai ^{[2
]}

Wu, Xiaofeng ^{[3
]}

Tang, Andrew Jack ^{[3
]}

Cheng, Xuebin ^{[4
]}

机构：

[1] Shanghai Jiao Tong Univ, Sch Aeronaut & Astronaut, Shanghai 200240, Peoples R China

[2] Beihang Univ, Sch Astronaut, Beijing 100191, Peoples R China

[3] Univ Sydney, Sch Aerosp Mech & Mechatron Engn, Sydney 2006, Australia

[4] China Aerosp Sci & Ind Corp, X Lab, Acad 2, Beijing 100854, Peoples R China

来源：

CHINESE JOURNAL OF AERONAUTICS | 2023年 / 36卷 / 02期

基金：

中国国家自然科学基金;

关键词：

LEO satellite networks; Mega constellation; Multi-objective optimization; Routing algorithm; Reinforcement learning; PROTOCOL;

D O I：

10.1016/j.cja.2022.06.021

中图分类号：

V [航空、航天];

学科分类号：

08 ; 0825 ;

摘要：

Recently, mega Low Earth Orbit (LEO) Satellite Network (LSN) systems have gained more and more attention due to low latency, broadband communications and global coverage for ground users. One of the primary challenges for LSN systems with inter-satellite links is the routing strategy calculation and maintenance, due to LSN constellation scale and dynamic network topology feature. In order to seek an efficient routing strategy, a Q-learning-based dynamic distributed Routing scheme for LSNs (QRLSN) is proposed in this paper. To achieve low end-toend delay and low network traffic overhead load in LSNs, QRLSN adopts a multi-objective optimization method to find the optimal next hop for forwarding data packets. Experimental results demonstrate that the proposed scheme can effectively discover the initial routing strategy and provide long-term Quality of Service (QoS) optimization during the routing maintenance process. In addition, comparison results demonstrate that QRLSN is superior to the virtual-topology-based shortest path routing algorithm. (c) 2022 Chinese Society of Aeronautics and Astronautics. Production and hosting by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

引用

页码：284 / 291

页数：8

共 23 条

[1]

[Anonymous], 1993, INT C NEUR INF PROC

[2]

Chang HS, 1998, IEEE T VEH TECHNOL, V47, P1037, DOI 10.1109/25.704858

[3] Multiagent Meta-Reinforcement Learning for Adaptive Multipath Routing Optimization [J].

Chen, Long ;

Hu, Bin ;

Guan, Zhi-Hong ;

Zhao, Lian ;

Shen, Xuemin .

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2022, 33 (10) :5374-5386

[4]

Choi SPM, 1996, ADV NEUR IN, V8, P945

[5] A technical comparison of three low earth orbit satellite constellation systems to provide global broadband [J].

del Portillo, Inigo ;

Cameron, Bruce G. ;

Crawley, Edward F. .

ACTA ASTRONAUTICA, 2019, 159 :123-135

[6] A distributed routing algorithm for datagram traffic in LEO satellite networks [J].

Ekici, E ;

Akyildiz, IF ;

Bender, MD .

IEEE-ACM TRANSACTIONS ON NETWORKING, 2001, 9 (02) :137-147

[7] QELAR: A Machine-Learning-Based Adaptive Routing Protocol for Energy-Efficient and Lifetime-Extended Underwater Sensor Networks [J].

Hu, Tiansi ;

Fei, Yunsi .

IEEE TRANSACTIONS ON MOBILE COMPUTING, 2010, 9 (06) :796-809

[8] Routing Algorithm with Virtual Topology Toward to Huge Numbers of LEO Mobile Satellite Network Based on SDN [J].

Jia, Min ;

Zhu, Siyu ;

Wang, Linfang ;

Guo, Qing ;

Wang, Haitao ;

Liu, Zhihui .

MOBILE NETWORKS & APPLICATIONS, 2018, 23 (02) :285-300

[9] QGeo: Q-Learning-Based Geographic Ad Hoc Routing Protocol for Unmanned Robotic Networks [J].

Jung, Woo-Sung ;

Yim, Jinhyuk ;

Ko, Young-Bae .

IEEE COMMUNICATIONS LETTERS, 2017, 21 (10) :2258-2261

[10] Temporal Netgrid Model-Based Dynamic Routing in Large-Scale Small Satellite Networks [J].

Li, Jian ;

Lu, Hancheng ;

Xue, Kaiping ;

Zhang, Yongdong .

IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, 2019, 68 (06) :6009-6021

← 1 2 3 →