Real-Time Event-Based Unsupervised Feature Consolidation and Tracking for Space Situational Awareness

Abstract

Earth orbit is a limited natural resource that hosts a vast range of vital space-based systems that support the international community's national, commercial and defence interests. This resource is rapidly becoming depleted with over-crowding in high demand orbital slots and a growing presence of space debris. We propose the Fast Iterative Extraction of Salient targets for Tracking Asynchronously (FIESTA) algorithm as a robust, real-time and reactive approach to optical Space Situational Awareness (SSA) using Event-Based Cameras (EBCs) to detect, localize, and track Resident Space Objects (RSOs) accurately and timely. We address the challenges of the asynchronous nature and high temporal resolution output of the EBC accurately, unsupervised and with few tune-able parameters using concepts established in the neuromorphic and conventional tracking literature. We show this algorithm is capable of highly accurate in-frame RSO velocity estimation and average sub-pixel localization in a simulated test environment to distinguish the capabilities of the EBC and optical setup from the proposed tracking system. This work is a fundamental step toward accurate end-to-end real-time optical event-based SSA, and developing the foundation for robust closed-form tracking evaluated using standardized tracking metrics.

Research of event cameras in space object detection

“…Subsequently, improvements were made to the method to enhance the robustness of tracking, making the algorithm easier to implement on sensor platforms in both spatial and ground environments. Reference [34] proposed an asynchronous tracking algorithm for fast iterative extraction of salient targets (FIESTA), which uses a feature extraction (FEAST) network based on adaptive selection threshold for real-time unsupervised feature extraction and merging. Through an improved asynchronous probability data association (PDA) tracker, the state and covariance of the target in the field of view are estimated.…”

Observations of Naturally Occurring Lightning with Event-Based Vision Sensors

“…In trackingby-detection, the output of a detector such as a clustering or filtering algorithm is also considered a track measurement. To estimate the state of a target, track measurements are associated to known or potential targets, which are used to infer desired target state qualities including position, speed and acceleration using Bayesian techniques such as Kalman Filtering (Kalman, 1960), Particle Filtering (Del Moral, 1997) or a general Markov-Bayes recursion (Ralph et al, 2022). Tracking algorithms generally use a track manager to maintain unique track identifiers and the life cycle of tracks.…”

TactiGraph: An Asynchronous Graph Neural Network for Contact Angle Prediction Using Neuromorphic Vision-Based Tactile Sensing

“…The pixel-wise intensity changes, called events or spikes, are recorded at a temporal resolution on the order of microseconds. Event-based cameras have been applied in autonomous drone racing [38], space imaging [39], space exploration [40], automated drilling [41], and visual servoing [42,43]. Neuromorphic cameras' fast update rate, along with their high dynamic range (140 dB compared to conventional cameras with 60 dB [44]) and low power consumption, make them apt for robotics tasks [45].…”