A Generic Framework for the Development of Geospatial Processing Pipelines on Clusters

Abstract

The amount of remote sensing data available to applications is constantly growing due to the rise of very-high-resolution sensors and short repeat cycle satellites. Consequently, tackling computational complexity in Earth Observation information extraction is rising as a major challenge. Resorting to High Performance Computing (HPC) is becoming a common practice, since it provides environments and programming facilities able to speed-up processes. In particular, clusters are flexible, cost-effective systems able to perform data-intensive tasks ideally fulfilling any computational requirement. However, their use typically implies a significant coding effort to build proper implementations of specific processing pipelines. This paper presents a generic framework for the development of RS images processing applications targeting cluster computing. It is based on common open sources libraries, and leverages the parallelization of a wide variety of image processing pipelines in a transparent way. Performances on typical RS tasks implemented using the proposed framework demonstrate a great potential for the effective and timely processing of large amount of data.

Biomedical Signal Acquisition Using Sensors under the Paradigm of Parallel Computing

“…The classification of Parallel paradigm in some computers is directly related to the level of parallel tasks that their hardware can support: multicore and multithreaded computers have multiple elements of processing on a single machine, while clusters or distributed computing employ multiple computers to work on the same task [35].…”

Orfeo ToolBox: open source processing of remote sensing images

“…But as the dataset gets bigger, applications generally moves to an HPC architecture sharing many similar nodes with a shared high-bandwidth storage, and Orfeo ToolBox can also do that. It allows for MPI [3] parallel processing, meaning that the whole pipeline will be replicated across nodes which will produce a piece of the resulting image [26]. An example of use of this capability is the pan-sharpening of a whole Pleiades image (1.6 gigapixels).…”

A Framework for Remote Sensing Images Processing Using Deep Learning Techniques

“…The Orfeo ToolBox (OTB) is a library for RS image processing, built on top of an application development framework widely used in medical image processing, the Insight Toolkit (ITK) [24]. The machine learning framework of OTB is able to process large datasets at continental scale for land mapping [25] and benefits from High Performance Computing (HPC) architectures like clusters [26]. TensorFlow (TF) is a library for dataflow programming.…”