CubeSat Simulation and Detection using Monocular Camera Images and Convolutional Neural Networks

Shi, Jianfeng; Ulrich, Steve; Ruel, Stephane

doi:10.2514/6.2018-1604

Cited by 19 publications

(18 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The implementation of CNNs for monocular pose estimation in space has already become an attractive solution in recent years [10][11][12], also thanks to the creation of the Spacecraft PosE Estimation Dataset (SPEED) [11], a database of highly representative synthetic images of PRISMA's TANGO spacecraft made publicly available by Stanford's Space Rendezvous Laboratory (SLAB) applicable to train and test different network architectures. One of the main advantages of CNNs over standard feature-based algorithms for relative pose estimation [3,13,14] is an increase in the robustness under adverse illumination condition, as well as a reduction in the computational complexity.…”

Section: Introductionmentioning

confidence: 99%

CNN-Based Pose Estimation System for Close-Proximity Operations Around Uncooperative Spacecraft

Cassinis

Fonod

Gill

et al. 2020

AIAA Scitech 2020 Forum

View full text Add to dashboard Cite

This paper introduces a novel framework which combines a Convolutional Neural Network (CNN) for feature detection with a Covariant Efficient Procrustes Perspective-n-Points (CEPPnP) solver and an Extended Kalman Filter (EKF) to enable robust monocular pose estimation for close-proximity operations around an uncooperative spacecraft. The relative pose estimation of an inactive spacecraft by an active servicer spacecraft is a critical task in the design of current and planned space missions, due to its relevance for close-proximity operations, such as In-Orbit Servicing and Active Debris Removal. The main contribution of this work stands in deriving statistical information from the Image Processing step, by associating a covariance matrix to the heatmaps returned by the CNN for each detected feature. This information is included in the CEPPnP to improve the accuracy of the pose estimation step during filter initialization. The derived measurement covariance matrix is used in a tightlycoupled EKF to allow an enhanced representation of the measurements error from the feature detection step. This increases the filter robustness in case of inaccurate CNN detections. The proposed method is capable of returning reliable estimates of the relative pose as well as of the relative translational and rotational velocities, under adverse illumination conditions and partial occultation of the target. Synthetic 2D images of the European Space Agency's Envisat spacecraft are used to generate datasets for training, validation and testing of the CNN. Likewise, the images are used to recreate representative close-proximity scenarios for the validation of the proposed method.

show abstract

Section: Introductionmentioning

confidence: 99%

CNN-Based Pose Estimation System for Close-Proximity Operations Around Uncooperative Spacecraft

Cassinis

Fonod

Gill

et al. 2020

AIAA Scitech 2020 Forum

View full text Add to dashboard Cite

show abstract

“…[34] for noncooperative spacecraft to solve a classification problem and return the relative pose of the space target associated to each image [14]. Shi et al use Inception-ResNet-V2 [35] and ResNet-101 [36], combined with an object detection engine [19], which improves their reliability. Sharma and D'Amico propose the SPN network [20,37] based on five convolutional layers, a Region Proposal Network (RPN) [38], and three fully-connected layers in order to generate the relative attitude of the target spacecraft.…”

Section: Pose Determination For Noncooperative Space Targetsmentioning

confidence: 99%

“…The CNNs have the advantages of higher robustness for adverse illumination condition and lower computational complexity over classical feature-based algorithms. Most of the CNN based pose determination methods [14,[17][18][19][20][21] are designed to solve a classification or regression problem and then return the relative pose of the space target, which is describeed in detail in the related work section. However, compared with CNNs based on keypoints, these regression or classification models are less generalized and are more easily interfered by low signal-to-noise-ratio, and the multiscale characteristics of space target images are often ignored.…”

Section: Introductionmentioning

confidence: 99%

An Improved Deep Keypoint Detection Network for Space Targets Pose Estimation

Song

Yang

et al. 2020

Remote Sensing

View full text Add to dashboard Cite

The on-board pose estimation of uncooperative target is an essential ability for close-proximity formation flying missions, on-orbit servicing, active debris removal and space exploration. However, the main issues of this research are: first, traditional pose determination algorithms result in a semantic gap and poor generalization abilities. Second, specific pose information cannot be accurately known in a complicated space target imaging environment. Deep learning methods can effectively solve these problems; thus, we propose a pose estimation algorithm that is based on deep learning. We use keypoints detection method to estimate the pose of space targets. For complicated space target imaging environment, we combined the high-resolution network with dilated convolution and online hard keypoint mining strategy. The improved network pays more attention to the obscured keypoints, has a larger receptive field, and improves the detection accuracy. Extensive experiments have been conducted and the results demonstrate that the proposed algorithms can effectively reduce the error rate of pose estimation and, compared with the related pose estimation methods, our proposed model has a higher detection accuracy and a lower pose determination error rate in the speed dataset.

show abstract

“…This was performed by means of transfer learning on the last three fully-connected layers. Shi et al [52] used two state-of-the-art CNNs, namely Inception-ResNet-V2 [53] and ResNet-101 [54], in combination with an object detection engine [55] to improve their reliability. Synthetic images generated in the 3DS-Max software were used in combination with real images to train and test the two networks, specifically 400 and 100 images, of which 8% were real images, were used for training and testing the networks, respectively.…”

Section: Cnn-based Pose Estimationmentioning

confidence: 99%

“…Also, larger datasets shall be considered for a comprehensive comparative assessment of the CNN architecture with the conventional pose determination architectures. Furthermore, assumptions on the illumination environment, target texture and reflectance properties shall be investigated to increase the robustness of the pose estimation, and different CNNs, such as the GoogLeNet, the ResNets and the DenseNet, shall be traded-off with respect to computational time and accuracy in the pose estimation, following the promising results reported in [52] for the Inception-ResNet-V2…”

Section: Cnn-based Pose Estimationmentioning

confidence: 99%

Review of the robustness and applicability of monocular pose estimation systems for relative navigation with an uncooperative spacecraft

Cassinis

Fonod

Gill

2019

Progress in Aerospace Sciences

View full text Add to dashboard Cite

The relative pose estimation of an inactive target by an active servicer spacecraft is a critical task in the design of current and planned space missions, due to its relevance for close-proximity operations, i.e. the rendezvous with a space debris and/or in-orbit servicing. Pose estimation systems based solely on a monocular camera are recently becoming an attractive alternative to systems based on active sensors or stereo cameras, due to their reduced mass, power consumption and system complexity. In this framework, a review of the robustness and applicability of monocular systems for the pose estimation of an uncooperative spacecraft is provided. Special focus is put on the advantages of multispectral monocular systems as well as on the improved robustness of novel image processing schemes and pose estimation solvers. The limitations and drawbacks of the validation of current pose estimation schemes with synthetic images are further discussed, together with the critical trade-offs for the selection of visual-based navigation filters. The state-of-the-art techniques are analyzed in order to provide an insight into the limitations involved under adverse illumination and orbit scenarios, high image contrast, background noise, and low signal-to-noise ratio, which characterize actual space imagery, and which could jeopardize the image processing algorithms and affect the pose estimation accuracy as well as the navigation filter's robustness. Specifically, a comparative

show abstract

CubeSat Simulation and Detection using Monocular Camera Images and Convolutional Neural Networks

Cited by 19 publications

References 3 publications

CNN-Based Pose Estimation System for Close-Proximity Operations Around Uncooperative Spacecraft

CNN-Based Pose Estimation System for Close-Proximity Operations Around Uncooperative Spacecraft

An Improved Deep Keypoint Detection Network for Space Targets Pose Estimation

Review of the robustness and applicability of monocular pose estimation systems for relative navigation with an uncooperative spacecraft

Contact Info

Product

Resources

About