Optimization of Active Debris Removal Missions with Multiple Targets

A Reinforcement Learning Scheme for Active Multi-Debris Removal Mission Planning With Modified Upper Confidence Bound Tree Search

“…Expanding mode 1 is uniformly random picking, and expanding mode 2 is picking under the distribution p. The illegal actions in Equ. (18) are filtered while expanding the tree.…”

“…Note that the percentage after the action represents its chosen possibility, and the layer represents the depth of the node in the search tree. Continue the search to 1000 more episodes, the stored best solution is changed to [1,30,18,37,57]. The average reward 7498117/1500000= 4.9987 on the root node which is close to the optimal reward of 5.…”

“…As the ADR mission planning technique is developed from 2010s, most researchers minimized ∆v cost of fixed length of removal sequence under particular dynamic model assumptions, such as Barbee et al [14] is under co-elliptic rendezvous strategy, Cerf [13], [15] is under drift-orbit transfer strategy with J 2 perturbation considered, Braun et al [8] is under four different scenarios with J 2 perturbation considered, Yu te al. [16], [17] is under two-impulse rendezvous, Shen et al [18] is under four-impulse transfer strategy with J 2 perturbation considered. Meanwhile, some researches also minimized the total mission time additionally [9], [17], [19], [20].…”

“…Heuristic or meta-heuristic approaches are proposed for this high level transfer sequence optimization, such as exhaustive search [8], [16], [19], branch and bound [9], [13], [20], [25], Ant Colony Algorithm (ACO) [18], [26], Genetic Algorithm (GA) [2], [10], [11], [15] and heuristic greedy method [12], [14], [27]. However, for future autonomous spacecraft, the planning will be an action choosing procedure, which can produce the best actions constantly, and keep the exploring ability as well.…”

“…However, the orbit of debris in LEO faces complicated dynamics making it hard to get the expectation of state-action value in the RL model, not to mention generating the solutions. So, the dynamic model is assumed to be under J 2 perturbation [8], [18] and the drift orbit transfer strategy [10]- [12], [15] is utilized to access to the impulse cost. Under this circumstance, the high-level planning is possible to be solved as the reference for future rendezvous planning and on-orbit operation, and all the processes of the planning are regarded to be deterministic functions, such as time-dependent cost function, state transition function, and reward function.…”