Adopting the Hadoop Architecture to Process Satellite Pollution Big Data

Semlali, Badr-Eddine Boudriki; Amrani, Chaker El; Ortiz, Guadalupe

doi:10.20469/ijtes.5.40001-2

Cited by 12 publications

(12 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These investigations aim mainly to employ parallel computing via the integration and the exploitation of the hardware's capacity [22] to store and process RSBD inside distributed clusters, notably the Hadoop [23]. To optimize algorithms and the processing patterns [24], and to process RS data in streaming [25].…”

Section: Introductionmentioning

confidence: 99%

Big data and remote sensing: A new software of ingestion

Semlali

Amrani

2021

IJECE

View full text Add to dashboard Cite

Currently, remote sensing is widely used in environmental monitoring applications, mostly air quality mapping and climate change supervision. However, satellite sensors occur massive volumes of data in near-real-time, stored in multiple formats and are provided with high velocity and variety. Besides, the processing of satellite big data is challenging. Thus, this study aims to approve that satellite data are big data and proposes a new big data architecture for satellite data processing. The developed software is enabling an efficient remote sensing big data ingestion and preprocessing. As a result, the experiment results show that 86 percent of the unnecessary daily files are discarded with a data cleansing of 20 percent of the erroneous and inaccurate plots. The final output is integrated into the Hadoop system, especially the HDFS, HBase, and Hive, for extra calculation and processing.

show abstract

Section: Introductionmentioning

confidence: 99%

Big data and remote sensing: A new software of ingestion

Semlali

Amrani

2021

IJECE

View full text Add to dashboard Cite

show abstract

“…Unlike operations with vector objects, processing and editing of raster data is much easier to parallelize due to their localization during processing. Most operations process one or more pixels located close to each other, which makes these calculations as easy as possible to parallelize with little or no need to solve the problems of mutual matching of data blocks (Boudriki et al, 2019;Sharma et al, 2017;Sharma et al, 2020).…”

Section: Methodsmentioning

confidence: 99%

Using Systems of Parallel and Distributed Data Processing to Build Hydrological Models Based on Remote Sensing Data

Kolesnikov

Кикин

Panidi

et al. 2021

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

Abstract. The article describes the possibilities and advantages of using distributed systems in the processing and analysis of remote sensing data. The preparation and processing of various types of remote sensing data (multispectral satellite images, values of climatic indicators, elevation data), which will then be used to build a simulation model of a hydroelectric power plant, was chosen as the basic task for testing the chosen approach. The existing approaches with distributed processing of spatial data of various types (vector cartographic objects, raster data, point clouds, graphs) are analyzed. The description of the developed approach is given and the rationale for the choice of its components is made. The preprocessing operations that were performed on the used raster data are described. An approach to the problems of raster data segmentation based on libraries for distributed machine learning is considered. Comparison of the speed of working with data for various algorithms of machine learning and processing is given.

show abstract

“…As the information from the different satellites is enormous, and to avoid details irrelevant to this paper, the rest of the background focuses on satellites that provide environmental information and, in particular, useful to monitor air pollution through RS data [13]. We collected data from the Exploitation of Meteorological Satellites (EUMETSAT) via the Mediterranean Dialogue Earth Observatory (MDEO) ground station installed at the Abdelmalek Essaâdi University of Tangier in Morocco, the Earth Observation System Data and Information System (EOSDIS) of the National Aeronautics and Space Administration (NASA), the National Environmental Satellite, Data, and Information Service (NESDIS) of the National Oceanic and Atmospheric Administration (NOAA), and the Copernicus Open Access Hub (COAH) of the European Space Agency (ESA).…”

Section: Satellite Specificationsmentioning

confidence: 99%