Review of the robustness and applicability of monocular pose estimation systems for relative navigation with an uncooperative spacecraft

Cassinis, Lorenzo Pasqualetto; Fonod, Robert; Gill, Eberhard

doi:10.1016/j.paerosci.2019.05.008

Cited by 75 publications

(31 citation statements)

References 70 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Interested readers are referred to Ref. [4] for a recent overview of the current trends in monocular-based pose estimation systems. Notably, the pose estimation problem is in this case complicated by the fact that the target satellite is uncooperative, namely retained as not functional and/or not able to aid the relative navigation.…”

Section: Introductionmentioning

confidence: 99%

CNN-Based Pose Estimation System for Close-Proximity Operations Around Uncooperative Spacecraft

Cassinis

Fonod

Gill

et al. 2020

AIAA Scitech 2020 Forum

Self Cite

View full text Add to dashboard Cite

This paper introduces a novel framework which combines a Convolutional Neural Network (CNN) for feature detection with a Covariant Efficient Procrustes Perspective-n-Points (CEPPnP) solver and an Extended Kalman Filter (EKF) to enable robust monocular pose estimation for close-proximity operations around an uncooperative spacecraft. The relative pose estimation of an inactive spacecraft by an active servicer spacecraft is a critical task in the design of current and planned space missions, due to its relevance for close-proximity operations, such as In-Orbit Servicing and Active Debris Removal. The main contribution of this work stands in deriving statistical information from the Image Processing step, by associating a covariance matrix to the heatmaps returned by the CNN for each detected feature. This information is included in the CEPPnP to improve the accuracy of the pose estimation step during filter initialization. The derived measurement covariance matrix is used in a tightlycoupled EKF to allow an enhanced representation of the measurements error from the feature detection step. This increases the filter robustness in case of inaccurate CNN detections. The proposed method is capable of returning reliable estimates of the relative pose as well as of the relative translational and rotational velocities, under adverse illumination conditions and partial occultation of the target. Synthetic 2D images of the European Space Agency's Envisat spacecraft are used to generate datasets for training, validation and testing of the CNN. Likewise, the images are used to recreate representative close-proximity scenarios for the validation of the proposed method.

show abstract

Section: Introductionmentioning

confidence: 99%

CNN-Based Pose Estimation System for Close-Proximity Operations Around Uncooperative Spacecraft

Cassinis

Fonod

Gill

et al. 2020

AIAA Scitech 2020 Forum

Self Cite

View full text Add to dashboard Cite

show abstract

“…The complete camera network provides a 360° coverage for no dead-zone awareness. As for the relative navigation algorithms, DL-based methods are the new trend to replace marker-based methods, which usually suffer from reduced robustness in harsh lighting conditions (Pasqualetto Cassinis et al, 2019 ). The 7-DOF articulated robot arm in the combined AI GNC system is inspired by the European Robotic Arm (ERA) (Boumans and Heemskerk, 1998 ).…”

Section: The Intelligent Gnc System Designmentioning

confidence: 99%

Intelligent Spacecraft Visual GNC Architecture With the State-Of-the-Art AI Components for On-Orbit Manipulation

et al. 2021

View full text Add to dashboard Cite

Conventional spacecraft Guidance, Navigation, and Control (GNC) architectures have been designed to receive and execute commands from ground control with minimal automation and autonomy onboard spacecraft. In contrast, Artificial Intelligence (AI)-based systems can allow real-time decision-making by considering system information that is difficult to model and incorporate in the conventional decision-making process involving ground control or human operators. With growing interests in on-orbit services with manipulation, the conventional GNC faces numerous challenges in adapting to a wide range of possible scenarios, such as removing unknown debris, potentially addressed using emerging AI-enabled robotic technologies. However, a complete paradigm shift may need years' efforts. As an intermediate solution, we introduce a novel visual GNC system with two state-of-the-art AI modules to replace the corresponding functions in the conventional GNC system for on-orbit manipulation. The AI components are as follows: (i) A Deep Learning (DL)-based pose estimation algorithm that can estimate a target's pose from two-dimensional images using a pre-trained neural network without requiring any prior information on the dynamics or state of the target. (ii) A technique for modeling and controlling space robot manipulator trajectories using probabilistic modeling and reproduction to previously unseen situations to avoid complex trajectory optimizations on board. This also minimizes the attitude disturbances of spacecraft induced on it due to the motion of the robot arm. This architecture uses a centralized camera network as the main sensor, and the trajectory learning module of the 7 degrees of freedom robotic arm is integrated into the GNC system. The intelligent visual GNC system is demonstrated by simulation of a conceptual mission—AISAT. The mission is a micro-satellite to carry out on-orbit manipulation around a non-cooperative CubeSat. The simulation shows how the GNC system works in discrete-time simulation with the control and trajectory planning are generated in Matlab/Simulink. The physics rendering engine, Eevee, renders the whole simulation to provide a graphic realism for the DL pose estimation. In the end, the testbeds developed to evaluate and demonstrate the GNC system are also introduced. The novel intelligent GNC system can be a stepping stone toward future fully autonomous orbital robot systems.

show abstract

“…Because the monocular camera has the advantages of reduced mass, power consumption and system complexity, it has become an attractive alternative to LIDAR [29][30][31] or stereo cameras [32] in pose determination systems during close-proximity operation. Pasqualetto Cassinis et al analyze the robustness and applicability of monocular pose estimation for noncooperative targets [7]. Sharma and D'Amico compare and evaluate the initial attitude estimation techniques of monocular navigation [33].…”

Section: Pose Determination For Noncooperative Space Targetsmentioning

confidence: 99%

“…Because little knowledge regarding the kinematic characteristics of the target is available in noncooperative maneuvers before missions, the rendezvous and docking trajectory must be generated on-board while using the current state estimates during the proximity operations. Pose estimation that is based on monocular cameras has the advantages of rapid pose determination under mass requirements and low power, while stereo cameras and LIDAR sensors are less convenient and less flexible in terms of mass, power consumption, and operation range [7]. Thus, monocular cameras based pose determination systems are becoming an attractive option.…”

Section: Introductionmentioning

confidence: 99%

An Improved Deep Keypoint Detection Network for Space Targets Pose Estimation

Song

Yang

et al. 2020

Remote Sensing

View full text Add to dashboard Cite

The on-board pose estimation of uncooperative target is an essential ability for close-proximity formation flying missions, on-orbit servicing, active debris removal and space exploration. However, the main issues of this research are: first, traditional pose determination algorithms result in a semantic gap and poor generalization abilities. Second, specific pose information cannot be accurately known in a complicated space target imaging environment. Deep learning methods can effectively solve these problems; thus, we propose a pose estimation algorithm that is based on deep learning. We use keypoints detection method to estimate the pose of space targets. For complicated space target imaging environment, we combined the high-resolution network with dilated convolution and online hard keypoint mining strategy. The improved network pays more attention to the obscured keypoints, has a larger receptive field, and improves the detection accuracy. Extensive experiments have been conducted and the results demonstrate that the proposed algorithms can effectively reduce the error rate of pose estimation and, compared with the related pose estimation methods, our proposed model has a higher detection accuracy and a lower pose determination error rate in the speed dataset.

show abstract

Review of the robustness and applicability of monocular pose estimation systems for relative navigation with an uncooperative spacecraft

Cited by 75 publications

References 70 publications

CNN-Based Pose Estimation System for Close-Proximity Operations Around Uncooperative Spacecraft

CNN-Based Pose Estimation System for Close-Proximity Operations Around Uncooperative Spacecraft

Intelligent Spacecraft Visual GNC Architecture With the State-Of-the-Art AI Components for On-Orbit Manipulation

An Improved Deep Keypoint Detection Network for Space Targets Pose Estimation

Contact Info

Product

Resources

About