Autonomous Small Body Science Operations Using Reinforcement Learning

被引:0
作者
Herrmann, Adam [1 ]
Schaub, Hanspeter [1 ]
机构
[1] Univ Colorado, Ann & HJ Smead Dept Aerosp Engn Sci, 3775 Discovery Dr,AERO 446, Boulder, CO 80303 USA
来源
JOURNAL OF AEROSPACE INFORMATION SYSTEMS | 2024年 / 21卷 / 10期
基金
美国国家科学基金会;
关键词
Reinforcement Learning; Markov Decision Process; Computer Storage Devices; Satellite Broadcasting; Asteroids; Unscented Kalman Filter; Optimal Control Theory; Deep Space Network; Planets; Guidance; Navigation; and Control Systems; GUIDANCE; ASTEROIDS; LIDAR;
D O I
10.2514/1.I011376
中图分类号
V [航空、航天];
学科分类号
08 ; 0825 ;
摘要
This work studies the feasibility of using reinforcement learning for small body science operations subject to resource constraints. Two mission scenarios are considered. In the first scenario, a spacecraft autonomously maneuvers between waypoints about a small body while performing science activities, such as mapping and imaging, and periodically downlinking data and managing on-board resources like battery charge, data buffer storage, and fuel usage. In the second scenario, the spacecraft periodically performs navigation updates to improve its state estimate, ensuring that the collected science is within the specified requirements. A Markov decision process formulation of the mission scenarios is formulated, and reinforcement learning is applied to solve the problem. A range of noisy observation types are tested, demonstrating that a fully observable formulation of the problem trained on direct observations of the state is robust to noisy measurements or a filtered state estimate. A decision-making agent is then trained to manage the state estimate by choosing when to take measurements, demonstrating that near-equivalent policies, in comparison to nominal problem formulation, can be trained with an optional navigation update. Finally, a demonstration is performed in which a ground station outage is simulated. The decision-making agent is shown to be robust to this outage, rapidly adjusting its plan to continue nominal operations.
引用
收藏
页码:865 / 884
页数:20
相关论文
共 61 条
[1]  
[Anonymous], 2023, IG23016 NASA OFF INS
[2]   Rosetta science operations in support of the Philae mission [J].
Ashman, Mike ;
Barthelemy, Maud ;
O'Rourke, Laurence ;
Almeida, Miguel ;
Altobelli, Nicolas ;
Costa Sitja, Marc ;
Garcia Beteta, Juan Jose ;
Geiger, Bernhard ;
Grieger, Bjorn ;
Heather, David ;
Hoofs, Raymond ;
Kuppers, Michael ;
Martin, Patrick ;
Moissl, Richard ;
Munoz Crego, Claudio ;
Perez-Ayucar, Miguel ;
Sanchez Suarez, Eduardo ;
Taylor, Matt ;
Vallat, Claire .
ACTA ASTRONAUTICA, 2016, 125 :41-64
[3]  
Bernard D., 1999, SPACE TECHNOLOGY C E, P28, DOI DOI 10.2514/6.1999-4512
[4]  
Bernard DE, 1998, 1998 IEEE AEROSPACE CONFERENCE PROCEEDINGS, VOL 2, P259, DOI 10.1109/AERO.1998.687914
[5]  
Bhaskaran Shyam., 1998, ORBIT DETERMINATION
[6]   Autonomous Imaging and Mapping of Small Bodies Using Deep Reinforcement Learning [J].
Chan, David M. ;
Agha-mohammadi, Ali-akbar .
2019 IEEE AEROSPACE CONFERENCE, 2019,
[7]   A mixed integer linear programming model for multi-satellite scheduling [J].
Chen, Xiaoyu ;
Reinelt, Gerhard ;
Dai, Guangming ;
Spitz, Andreas .
EUROPEAN JOURNAL OF OPERATIONAL RESEARCH, 2019, 275 (02) :694-707
[8]  
Chien S., 2005, J AEROSPACE COMPUTIN, P196, DOI [DOI 10.2514/1.12923, 10.2514/1.12923]
[9]   Onboard Autonomy on the Intelligent Payload Experiment CubeSat Mission [J].
Chien, Steve ;
Doubleday, Joshua ;
Thompson, David R. ;
Wagstaff, Kiri L. ;
Bellardo, John ;
Francis, Craig ;
Baumgarten, Eric ;
Williams, Austin ;
Yee, Edmund ;
Stanton, Eric ;
Piug-Suari, Jordi .
JOURNAL OF AEROSPACE INFORMATION SYSTEMS, 2017, 14 (06) :307-315
[10]  
Chien SteveA., 2010, P OF THE 20 INT C ON, P34, DOI [DOI 10.1609/ICAPS.V20I1.13410, 10.1609/icaps.v20i1.13410]