In-orbit demonstration of artificial intelligence applied to hyperspectral and thermal sensing from space

Esposito, Marco; Conticello, Simon; Pastena, Massimiliano; Domínguez, Bernardo Carnicero

doi:10.1117/12.2532262

Cited by 33 publications

(36 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

Section: The Cloudscout Case Studymentioning

confidence: 99%

“…The CloudScout project targets the development of the very first CNN-based algorithm to fly on board a satellite [21]. The CNN is exploited to perform cloud-detection by processing the hyperspectral images produced by the novel HyperScout-2 sensor on board the Phisat-1 6U CubeSat launched in September 2020 [38,39]. The algorithm classifies cloud-covered images and clear ones intending to discard unusable data and, thus, reduce the downlink bandwidth of the satellite [3].…”

Section: The Cloudscout Case Studymentioning

confidence: 99%

“…In this work, we present an FPGA-based hardware accelerator for the CloudScout Deep Neural Network [3]. This network was developed in the framework of the Cloud-Scout project [21], led by the Dutch company Cosine Measurement systems [22] and funded by the European Space Agency (ESA). The aim of this project was to realize a Convolutional Neural Network (CNN) for on board cloud detection of satellite hyperspectral images.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An FPGA-Based Hardware Accelerator for CNNs Inference on Board Satellites: Benchmarking with Myriad 2-Based Solution for the CloudScout Case Study

et al. 2021

View full text Add to dashboard Cite

In recent years, research in the space community has shown a growing interest in Artificial Intelligence (AI), mostly driven by systems miniaturization and commercial competition. In particular, the application of Deep Learning (DL) techniques on board Earth Observation (EO) satellites might lead to numerous advantages in terms of mitigation of downlink bandwidth constraints, costs, and increment of the satellite autonomy. In this framework, the CloudScout project, funded by the European Space Agency (ESA), represents the first time in-orbit demonstration of a Convolutional Neural Network (CNN) applied to hyperspectral images for cloud detection. The first instance of this use case has been done with an INTEL Myriad 2 VPU on board a CubeSat optimized for low cost, size, and power efficiency. Nevertheless, this solution introduces multiple drawbacks due to its design not specifically being for the space environment, thus limiting its applicability to short-lifetime Low Earth Orbit (LEO) applications. The current work provides a benchmark between the Myriad 2 and our custom hardware accelerator designed for Field Programmable Gate Arrays (FPGAs). The metrics used for comparison include inference time, power consumption, space qualification, and components. The obtained results show that the FPGA-based solution is characterized by a reduced inference time, and a higher possibility of customization, but at the cost of greater power consumption and a longer Time to Market. As a conclusion, the proposed approach might extend the potential market of DL-based solutions to long-term LEO or interplanetary exploration missions through deployment on space-qualified FPGAs, with a limited cost in energy efficiency.

show abstract

Section: The Cloudscout Case Studymentioning

confidence: 99%

Section: The Cloudscout Case Studymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An FPGA-Based Hardware Accelerator for CNNs Inference on Board Satellites: Benchmarking with Myriad 2-Based Solution for the CloudScout Case Study

et al. 2021

View full text Add to dashboard Cite

show abstract

“…The challenge for LOUPE is collapsing four-dimensional data (F, P, χ , and λ) onto a twodimensional detector, instantaneously for Earth's entire disc. The instrument will be built on cosine Remote Sensing's 5 HyperScout 6 hyperspectral imaging platform [41,42], space qualified and operating in Earth orbit for almost two and a half years, 7 which is based on CMOS and linear variable filter (LVF) technologies. Because apart from the Sun, the Earth is the brightest object in the sky as seen from the Moon, we can use a wide-field micro-lens array (MLA) instead of a traditional telescope objective system.…”

Section: Loupe Instrument Designmentioning

confidence: 99%

LOUPE: observing Earth from the Moon to prepare for detecting life on Earth-like exoplanets

Klindžić¹,

Stam

Snik

et al. 2020

Phil. Trans. R. Soc. A.

View full text Add to dashboard Cite

LOUPE, the Lunar Observatory for Unresolved Polarimetry of the Earth, is a small, robust spectro-polarimeter for observing the Earth as an exoplanet. Detecting Earth-like planets in stellar habitable zones is one of the key challenges of modern exoplanetary science. Characterizing such planets and searching for traces of life requires the direct detection of their signals. LOUPE provides unique spectral flux and polarization data of sunlight reflected by Earth, the only planet known to harbour life. These data will be used to test numerical codes to predict signals of Earth-like exoplanets, to test algorithms that retrieve planet properties, and to fine-tune the design and observational strategies of future space observatories. From the Moon, LOUPE will continuously see the entire Earth, enabling it to monitor the signal changes due to the planet’s daily rotation, weather patterns and seasons, across all phase angles. Here, we present both the science case and the technology behind LOUPE’s instrumental and mission design. This article is part of a discussion meeting issue ‘Astronomy from the Moon: the next decades’.

show abstract

“…We developed a process to analyze the compact spectral imaging system trade space with subpixel target detection as the focus application. The underlying assumption for the miniature systems designed using this process was that the data collected will be automatically processed and a part of initiatives to use artificial intelligence in-orbit [19].…”

Section: Motivationmentioning

confidence: 99%

Optimizing Requirements for a Compact Spaceborne Adaptive Spectral Imaging System in Subpixel Target Detection Applications

Han

Kerekes

Higbee

et al. 2020

IEEE J. Miniat. Air Space Syst.

View full text Add to dashboard Cite

We developed a process to provide design recommendations for compact spaceborne spectral imaging systems with adaptive band selection capabilities. Our focus application was subpixel target detection, and we analyzed a set of mission scenarios to find relationships in detection performance between selected parameters of interest. We used an analytic model to predict performance and generate trade curves, then simulated a scene to analyze potential operational effects on performance for the selected target and background combinations. Using these models, we predicted and assessed each scenario to provide recommendations for mission feasibility and system design. The parameters we selected for analysis were target fill fraction, noise, number of bands, and scene complexity to find critical points in the trade space and reach a set of recommendations. We examined the operational effects by simulating a realistic scenario and ensuring key real-world phenomena were captured within the spectral images. Our results produced recommendations for each mission and provided a proof of concept for a process to analyze designs of miniature spaceborne imaging systems.

show abstract

In-orbit demonstration of artificial intelligence applied to hyperspectral and thermal sensing from space

Cited by 33 publications

References 1 publication

An FPGA-Based Hardware Accelerator for CNNs Inference on Board Satellites: Benchmarking with Myriad 2-Based Solution for the CloudScout Case Study

An FPGA-Based Hardware Accelerator for CNNs Inference on Board Satellites: Benchmarking with Myriad 2-Based Solution for the CloudScout Case Study

LOUPE: observing Earth from the Moon to prepare for detecting life on Earth-like exoplanets

Optimizing Requirements for a Compact Spaceborne Adaptive Spectral Imaging System in Subpixel Target Detection Applications

Contact Info

Product

Resources

About