Data pipeline of a multi-spectral satellite experiment for object detection and artificial intelligence-based processing

Abstract

Recent developments in the military domain introduce the need to detect and track hypersonic glide vehicles in Earth's atmosphere.The Multispectral Object Sensing by Artificial Intelligence-processed Cameras (MOSAIC) experiment is part of the small-satellite ATHENE-1 of the Universität der Bundeswehr München. The primary scientific objective of MOSAIC is to demonstrate reliable detection, identification a nd t racking o f h ypersonic g lide v ehicles using primarily a cooled infrared camera and complementary a visual camera. To cope with a large amount of data from both high-resolution cameras in real-time, state-of-the-art computer vision on-board processing methods are used for detection and tracking. The secondary scientific objective is to investigate the efficiency and reliability of Artificial Intelligence (AI) based image processing algorithms and data compression for space a pplications. This is of particular importance given the high volumes and rates of data. The application of such algorithms requires a reliable and resource-efficient On -Board Co mputer (O BC) th at ca n wi thstand th e ha rsh sp ace environment. The approach outlined in this paper envisions a dedicated OBC to manage the AI-based experiments of the satellite, called the Artificial Intelligence capable On-Board Computer ( AI-OBC). The AI-OBC includes multiple hardware-based AI accelerators to meet the computational requirements and ensure real-time processing for object detection and tracking. This paper describes the structure of the data processing pipeline and includes the AI-OBC architecture with its connections to both the cameras and the platform's OBC. Further, the study discusses the training and validation steps of the intended use-cases.

Development and testing of a multi-spectral and multi-field of view imaging system for space-based signature detection and tracking of hypersonic vehicles

“…Considering the high velocity of HGVs and the aim of tracking them on a real-time basis by slewing the platform, high speed on-board processing is another requirement. To fulfill also secondary usecases of MOSAIC 12 and moreover to investigate the feasibility of HGV detection within a complementary Visual (VIS) spectral range, MOSAIC's overall architecture is expanded by an additional high-resolution VIS camera. Given all these challenges and likewise given the restrictions and limitations of a small-satellite, the need for state-of-the-art components soon becomes clear.…”

“…The AI-OBC consists of a frame-grabber connected to the IR detector, a Graphics Processing Unit (GPU) to enable Artificial Intelligence (AI)-based on-board image processing and a Mass Memory Unit (MMU) to store all experiment related data. For further details on the AI-OBC the reader is referred to Müller et al 12 .…”

“…It consists of a 25 M P x high resolution Complementary Metal-Oxide-Semiconductor (CMOS) Focal Plane Array (FPA). Enabling various AI-based use-cases 12 that rely on color images, a polychromatic sensor is utilized generating Red, Green, Blue (RGB) images. All Telemetry (TM) and Telecommand (TC) communication, the power supply, as well as the transfer of scientific data is conducted via a direct link to the AI-OBC's GPU.…”

“…An exemplary validation scenario foresees the detection of stationary targets such as fires on the Earth's surface. 12 This however, would require a different spectral band of observation compared to HGVs to overcome the atmospheric absorption at lower altitudes. The last of the six filters in total is a blind filter with a black surface.…”