Relative control of an underactuated spacecraft using reinforcement learning

Khoroshylov, S.V.; Redka, M.O.

doi:10.15407/itm2020.04.043

Cited by 5 publications

(4 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Reference [27] approximates optimal relative control of an underactuated SC using RL and studies the influence of various factors on the performance of such a solution. The problem of in-plane SC relative control using only control actions applied only intrack direction is considered.…”

Section: Relative Controlmentioning

confidence: 99%

See 1 more Smart Citation

Deep learning for spacecraft guidance, navigation, and control

Khoroshylov¹,

Redka²

2021

Kosm. nauka tehnol.

Self Cite

View full text Add to dashboard Cite

The advances in deep learning have revolutionized the field of artificial intelligence, demonstrating the ability to create autonomous systems with a high level of understanding of the environments where they operate. These advances, as well as new tasks and requirements in space exploration, have led to an increased interest in these deep learning methods among space scientists and practitioners. The goal of this review article is to analyze the latest advances in deep learning for navigation, guidance, and control problems in space. The problems of controlling the attitude and relative motion of spacecraft are considered for both traditional and new missions, such as orbital service. The results obtained using these methods for landing and hovering operations considering missions to the Moon, Mars, and asteroids are also analyzed. Both supervised and reinforcement learning are used to solve such problems based on various architectures of artificial neural networks, including convolutional and recurrent ones. The possibility of using deep learning together with methods of control theory is analyzed to solve the considered problems more efficiently. The difficulties that limit the application of the reviewed methods for space applications are highlighted. The necessary research directions for solving these problems are indicated.

show abstract

Section: Relative Controlmentioning

confidence: 99%

“…The RL-based search for control actions is made using the policy iteration algorithm [27]. This algorithm is implemented using the "actor-critic" architecture.…”

Section: Relative Controlmentioning

confidence: 99%

Deep learning for spacecraft guidance, navigation, and control

Khoroshylov¹,

Redka²

2021

Kosm. nauka tehnol.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The article [14] presents an approximation of the optimal relative control for the underactuated spacecraft using the RL and the study of the influence of various factors on the performance of such a solution. This approach allows finding close to optimal control algorithms as a result of the interaction of the control system with the plant using the reinforcement signal to estimate the performance of the control actions.…”

Section: Introductionmentioning

confidence: 99%

Untitled

2024

Kosm. nauka tehnol.

View full text Add to dashboard Cite

The article investigates the task of spacecraft relative control using reactive actuators, the output of which has two states, "on" or "off". For cases where the resolution of the thrusters does not provide an accurate approximation of linear control laws using a pulse-width thrust modulator, the possibility of applying reinforcement learning methods for direct finding of control laws that map the state vector and the on-off thruster commands has been investigated. To implement such an approach, a model of controlled relative motion of two satellites in the form of a Markov decision process was obtained. The intelligent agent is presented in the form of "actor" and "critic" neural networks, and the architecture of these modules is defined. It is proposed to use a cost function with variable weights of control actions, which allows for optimizing the number of thruster firings explicitly. To improve the control performance, it is proposed to use an extended input vector for the "actor" and "critic" neural networks of the intelligent agent, which, in addition to the state vector, also includes information about the control action on the previous control step and the control step number. To reduce the training time, the agent was pre-trained on the data obtained using conventional control algorithms. Numerical results demonstrate that the reinforcement learning methodology allows the agent to outperform the results provided by the linear controller with the pulse-width modulator in terms of control accuracy, response time, and number of thruster firings.

show abstract

“…The conventional approach for this task results in computationally complex algorithms since they are based on integration of the elementary forces over the SDO surface. This work demonstrates that these tasks can be addressed using supervised [3] and reinforcement learning (RL) [4,5] techniques. Determination of the ion beam impact on a space debris object using convolutional neural networks.…”

mentioning

confidence: 99%

Deep Learning for Space Debris Removal

Serhii,

Mikhailo

2024

ITMM

View full text Add to dashboard Cite

The advances in deep learning have revolutionized the field of artificial intelligence. These advances, as well as new tasks and requirements in space exploration, have led to an increased interest in these deep learning methods among space scientists and practitioners. The problems of controlling the attitude and relative motion of spacecraft are considered for both traditional and new missions, such as contactless space debris removal. Both supervised and reinforcement learning is used to solve such problems based on various architectures of artificial neural networks, including convolutional ones. The possibility of using deep learning together with methods of control theory is analyzed to solve the considered problems more efficiently. The difficulties that limit the application of these methods for space applications are highlighted. The necessary research directions for solving these problems are indicated.

show abstract

Relative control of an underactuated spacecraft using reinforcement learning

Cited by 5 publications

References 16 publications

Deep learning for spacecraft guidance, navigation, and control

Deep learning for spacecraft guidance, navigation, and control

Untitled

Deep Learning for Space Debris Removal

Contact Info

Product

Resources

About