Pose estimation for non-cooperative spacecraft rendezvous using convolutional neural networks

Abstract

This work introduces the Spacecraft Pose Network (SPN) for on-board estimation of the pose, i.e., the relative position and attitude, of a known non-cooperative spacecraft using monocular vision. In contrast to other state-of-the-art pose estimation approaches for spaceborne applications, the SPN method does not require the formulation of hand-engineered features and only requires a single grayscale image to determine the pose of the spacecraft relative to the camera. The SPN method uses a Convolutional Neural Network (CNN) with three branches to solve the problems of relative attitude and relative position estimation. The first branch of the CNN bootstraps a state-of-the-art object detection algorithm to detect a 2D bounding box around the target spacecraft in the input image. The region inside the 2D bounding box is then used by the other two branches of the CNN to determine the relative attitude by initially classifying the input region into discrete coarse attitude labels before regressing to a finer estimate. The SPN method then uses a novel Gauss-Newton algorithm to estimate the relative position by using the constraints imposed by the detected 2D bounding box and the estimated relative attitude. The secondary contribution of this work is the generation of the Spacecraft PosE Estimation Dataset (SPEED), which is used to train and evaluate the performance of the SPN method. SPEED consists of synthetic as well as actual camera images of a mock-up of the Tango spacecraft from the PRISMA mission. The synthetic images are created by fusing OpenGL-based renderings of the spacecraft's 3D model with actual images of the Earth captured by the Himawari-8 meteorological satellite. The actual camera images are created using a 7 degrees-of-freedom robotic arm, which positions and orients a vision-based sensor with respect to a full-scale mock-up of the Tango spacecraft. Custom illumination devices simulate the Earth albedo and Sun light with high fidelity to emulate the illumination conditions present in space. The SPN method, trained only on synthetic images, produces degree-level relative attitude error and cm-level relative position errors when evaluated on the actual camera images not used during training.

Satellite Pose Estimation with Deep Landmark Regression and Nonlinear Pose Refinement

“…We first compare against SPN [30]; Table 1 report the performance results. Our proposed method achieves superior performances in both object detection and pose estimation.…”

“…Mirroring the setting during proximity operations, the size, orientation and lighting condition of the satellite in the images vary significantly, e.g., the number of object pixels vary between 1k and 500k; see Figure 3 for an example. For more details of the dataset, see [30].…”

“…The Spacecraft Pose Network (SPN) [30] is the seminal work on the SPEED. SPN uses a hybrid of classification and regression neural networks for the pose estimation problem.…”

“…Due to the importance of the problem, the Advanced Concepts Team (ACT) at ESA recently held a benchmark competition called Kelvins Pose Estimation Challenge (KPEC) [3]; given images that depict a known satellite under different unknown poses (see Figure 1), estimate the pose of the satellite in each image. To develop their al- Figure 1: Sample images of the Tango satellite from SPEED [30]. Note the significant variations in object size, object orientation, background and lighting condition.…”

Robust Model-Based Monocular Pose Initialization for Noncooperative Spacecraft Rendezvous

“…Due to these advantages, navigation systems employing monocular cameras have been proposed in order to enable rapid pose estimation and tracking in close range: up to a few centimeters to the target using limited mass and power resources [8][9][10][11][12][13][14][15]. Typically, these systems employ an image processing subsystem that identifies the visible target's features in the monocular image followed by a dedicated pose solver and an extended Kalman filter.…”