导航

ACTA AERONAUTICAET ASTRONAUTICA SINICA ›› 2021, Vol. 42 ›› Issue (4): 524354-524354.doi: 10.7527/S1000-6893.2020.24354

• Electronics and Electrical Engineering and Control • Previous Articles     Next Articles

Heuristic enhanced reinforcement learning method for large-scale multi-debris active removal mission planning

YANG Jianan1, HOU Xiaolei1, HU Yu Hen2, LIU Yong1, PAN Quan1, FENG Qian1   

  1. 1. College of Automation, Northwestern Polytechnical University, Xi'an 710129, China;
    2. Department of Electrical and Computer Engineering, University of Wisconsin-Madison, Madison 53706, USA
  • Received:2020-06-02 Revised:2020-09-12 Published:2020-10-10
  • Supported by:
    National Natural Science Foundation of China (61703343, 61790552); Natural Science Foundation of Shaanxi (2018JQ6070); The Fundamental Research Funds for Central Universities (3102018JCC003)

Abstract: Vigorous development of the space industry leads to a nonnegligible space debris threat to future space activities. The Active multi-Debris Removal (ADR) technology has become an indispensable means to alleviate this situation. Aiming at the large-scale multi-debris active removal mission planning problem, a Reinforcement Learning (RL) planning scheme is first proposed based on the maximal-reward optimization model for the ADR problem, and the state, action, and reward function of this problem are defined according to the RL framework. Based on an efficient heuristics method, a specialized Monte Carlo Tree Search (MCTS) algorithm is then presented, with the Monte Carlo Tree Search as the core structure and efficient heuristic operators and reinforcement learning iteration process. Finally, its effectiveness is tested in the large-scale complete Iridium 33 debris cloud. The results show that this method is superior to the original MCTS algorithm and the heuristic greedy algorithm.

Key words: active debris removal, mission planning, reinforcement learning, heuristic operator, Monte Carlo tree search

CLC Number: