The Parallel Image Processing Environment (PIPE): automated parallelization of satellite data analyses

Simpson, Joseph; McIntire, T.J.; Berg, Jared S.; Tsou, Yueh Lung

doi:10.1002/cpe.1058

Cited by 11 publications

(5 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Because it is a per-pixel time-series approach, this classification model analyzed a data volume substantially larger than U-Net in the prediction step. Since, within the scope of satellite data classification, this step is embarrassingly parallel [62], its execution time could be dramatically reduced through a high performance processing solution (e.g., in-house grid computing, Google Earth Engine).…”

Section: Discussionmentioning

confidence: 99%

Next Generation Mapping: Combining Deep Learning, Cloud Computing, and Big Remote Sensing Data

et al. 2019

View full text Add to dashboard Cite

The rapid growth of satellites orbiting the planet is generating massive amounts of data for Earth science applications. Concurrently, state-of-the-art deep-learning-based algorithms and cloud computing infrastructure have become available with a great potential to revolutionize the image processing of satellite remote sensing. Within this context, this study evaluated, based on thousands of PlanetScope images obtained over a 12-month period, the performance of three machine learning approaches (random forest, long short-term memory-LSTM, and U-Net). We applied these approaches to mapped pasturelands in a Central Brazil region. The deep learning algorithms were implemented using TensorFlow, while the random forest utilized the Google Earth Engine platform. The accuracy assessment presented F1 scores for U-Net, LSTM, and random forest of, respectively, 96.94%, 98.83%, and 95.53% in the validation data, and 94.06%, 87.97%, and 82.57% in the test data, indicating a better classification efficiency using the deep learning approaches. Although the use of deep learning algorithms depends on a high investment in calibration samples and the generalization of these methods requires further investigations, our results suggest that the neural network architectures developed in this study can be used to map large geographic regions that consider a wide variety of satellite data (e.g., PlanetScope, Sentinel-2, Landsat-8).

show abstract

Section: Discussionmentioning

confidence: 99%

Next Generation Mapping: Combining Deep Learning, Cloud Computing, and Big Remote Sensing Data

et al. 2019

View full text Add to dashboard Cite

show abstract

“…The DIAL-developed daemons optimize the execution of remotely sensed applications which typically are both computationally and I/O bound and simultaneously insulate the application and its developer from system issues (e.g., system complexity, memory management, disk management, I/O data transfers, product generation tracking). Optimal utilization of PIPE is best achieved by properly balancing the computational and I/O demands of a given application (see [7] for details).…”

Section: Discussionmentioning

confidence: 99%

“…Several types of messages are used. Inter-process/inter-daemon communication is totally transparent to the application developer (see [7] for details).…”

Section: Middleware Sublayer 2: Daemon Supported Automated User-tranmentioning

confidence: 99%

Parallel processing of remotely sensed data: Application to the ATSR-2 instrument

Simpson

McIntire

Berg

et al. 2007

Infrared Physics & Technology

View full text Add to dashboard Cite

“…In the example, the computations for a certain pixel, i.e. the pixel at spatial coordinates (5,3) in the original imagedenoted by f(5, 3)-need to originate from two processing elements since this pixel becomes a border pixel after spatial-domain partitioning. As a result, a communication overhead involving three N -dimensional pixel vectors (located in partition #2) is required in order to complete the kernel-based computation for the pixel f(5, 3) in partition #1.…”

Section: Data Partitioning Strategiesmentioning

confidence: 99%

“…In recent years, several efforts have been directed towards the incorporation of massively parallel computing models into remote sensing applications [5,6], especially with the advent of relatively cheap Beowulf clusters. For instance, the concept of Beowulf cluster originated at the Center of Excellence in Space and Data Information Sciences (CESDIS), located at NASA's Goddard Space Flight Center in Maryland [7].…”

Section: Introductionmentioning

confidence: 99%

Parallel processing of remotely sensed hyperspectral imagery: full‐pixel versus mixed‐pixel classification

Plaza

2008

Concurrency and Computation

View full text Add to dashboard Cite

SUMMARYThe rapid development of space and computer technologies allows for the possibility to store huge amounts of remotely sensed image data, collected using airborne and satellite instruments. In particular, NASA is continuously gathering high-dimensional image data with Earth observing hyperspectral sensors such as the Jet Propulsion Laboratory's airborne visible-infrared imaging spectrometer (AVIRIS), which measures reflected radiation in hundreds of narrow spectral bands at different wavelength channels for the same area on the surface of the Earth. The development of fast techniques for transforming massive amounts of hyperspectral data into scientific understanding is critical for space-based Earth science and planetary exploration. Despite the growing interest in hyperspectral imaging research, only a few efforts have been devoted to the design of parallel implementations in the literature, and detailed comparisons of standardized parallel hyperspectral algorithms are currently unavailable. This paper compares several existing and new parallel processing techniques for pure and mixed-pixel classification in hyperspectral imagery. The distinction of pure versus mixed-pixel analysis is linked to the considered application domain, and results from the very rich spectral information available from hyperspectral instruments. In some cases, such information allows image analysts to overcome the constraints imposed by limited spatial resolution. In most cases, however, the spectral bands collected by hyperspectral instruments have high statistical correlation, and efficient parallel techniques are required to reduce the dimensionality of the data while retaining the spectral information that allows for the separation of the classes. In order to address this issue, this paper also develops a new parallel feature extraction algorithm that integrates the spatial and spectral information. The proposed technique is evaluated (from the viewpoint of both classification accuracy and parallel performance) and compared with other parallel techniques for dimensionality reduction and classification in the context of three representative application case * Correspondence to: Antonio J. Plaza, Department of Technology of Computers and Communications, University of Extremadura, Avda. de la Universidad s/n, E-10071 Càceres, Spain. † E-mail: aplaza@unex.es studies: urban characterization, land-cover classification in agriculture, and mapping of geological features, using AVIRIS data sets with detailed ground-truth. Parallel performance is assessed using Thunderhead, a massively parallel Beowulf cluster at NASA's Goddard Space Flight Center. The detailed cross-validation of parallel algorithms conducted in this work may specifically help image analysts in selection of parallel algorithms for specific applications.

show abstract

The Parallel Image Processing Environment (PIPE): automated parallelization of satellite data analyses

Cited by 11 publications

References 24 publications

Next Generation Mapping: Combining Deep Learning, Cloud Computing, and Big Remote Sensing Data

Next Generation Mapping: Combining Deep Learning, Cloud Computing, and Big Remote Sensing Data

Parallel processing of remotely sensed data: Application to the ATSR-2 instrument

Parallel processing of remotely sensed hyperspectral imagery: full‐pixel versus mixed‐pixel classification

Contact Info

Product

Resources

About