SAT-Hadoop-Processor: A Distributed Remote Sensing Big Data Processing Software for Earth Observation Applications

Semlali, Badr-Eddine Boudriki; Freitag, Félix

doi:10.3390/app112210610

Cited by 11 publications

(6 citation statements)

References 35 publications

(43 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Only three earthquakes with M w > 6 have been visualized because they clearly show the correlation. The preprocessing of this massive volume of data was accomplished efficiently using the SAT-ETL-Integrator [26] and SAT-Hadoop-Processor software [27].…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

First Results on the Systematic Search of Land Surface Temperature Anomalies as Earthquakes Precursors

et al. 2023

Self Cite

View full text Add to dashboard Cite

Every year, earthquakes cause thousands of casualties and high economic losses. For example, in the time frame from 1998 to 2018, the total number of casualties due to earthquakes was larger than 846 thousand people, and the recorded economic losses were about USD 661 billion. At present, there are no earthquake precursors that can be used to trigger a warning. However, some studies have analyzed land surface temperature (LST) anomalies as a potential earthquake precursor. In this study, a large database of global LST data from the Geostationary Operational Environmental Satellite (GOES) and AQUA satellites during the whole year 2020 has been used to study the LST anomalies in the areas affected by earthquakes. A total of 1350 earthquakes with a magnitude larger than M4 were analyzed. Two methods widely used in the literature have been used to detect LST anomalies in the detrended LST time series: the interquartile (IQT) method and the standard deviation (STD). To the authors’ knowledge, it is the first time that the confusion matrix (CM), the receiver operating characteristic curve (ROC), and some other figures of merit (FoM) are used to assess and optimize the performance of the methods, and to select the optimum combination that could be used as a proxy for their occurrence. A positive anomaly was found a few days before the studied earthquakes, followed by the LST decrease after the event. Further studies over larger regions and more extended periods will be needed to consolidate these encouraging results.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Table 1 shows the satellites and sensors used in this study to provide the LST data. Their spatial resolution (SPR) ranges between 1 and 4 km, and their temporal resolution (TMP) resolution is between 1 h and 12 h, depending on the orbit [27]. In this study, global data is collected for 2020 [28].…”

Section: Input Datamentioning

confidence: 99%

First Results on the Systematic Search of Land Surface Temperature Anomalies as Earthquakes Precursors

et al. 2023

Self Cite

View full text Add to dashboard Cite

show abstract

“…The total size of the processed data is 12 TB. Parallel and distributed tools were applied to optimize the execution time by 90% with SAT-Hadoop-Processor [40], the distributed version of the SAT-ETL-Integrator [41].…”

Section: Figure 1 the General Architecture Of S4 Data Processing And ...mentioning

confidence: 99%

On the Correlation Between Earthquakes and Prior Ionospheric Scintillations Over the Ocean: A Study Using GNSS-R Data Between 2017 and 2021

Boudriki Semlali,

Molina,

Park

et al. 2024

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

Self Cite

View full text Add to dashboard Cite

From 1980 to 2021, earthquakes have caused more than 846,000 casualties and about US$ 661 billion in economic losses. At present, there are no reliable earthquake precursors to generate alerts. Currently, the link between earthquakes and Total Electron Content (TEC) variations measured by Global Navigation Satellite Systems (GNSS) monitoring ground stations has been studied. However, GNSS ground-based monitoring stations are irregularly disseminated around the globe with significant gaps, particularly in the ocean's regions. In this study, we analyze ionospheric intensity scintillation anomalies as potential proxies of earthquakes. NASA CYGNSS GNSS-R (Reflectometry) acquired by CYGNSS/DDMI (Delay Doppler Mapping Instrument) from 2017 to 2021 have been used to calculate and analyze the anomalies in the S4 parameter over ocean areas affected by earthquakes. More than 30,000 ocean earthquakes within ±40° in latitude and with a magnitude larger than M4 have been examined.The daily planetary geomagnetic index Kp was used to discard data that may be disturbed due to space weather conditions. Additionally, a daily sea wind speed mask was used to eliminate sea states that impact the signal reflectivity. The Standard Deviation (STD) and the Interquartile (IQT) time series methods have been used to detect these S4 anomalies. The confusion matrix, the Receiver Operating Curve (ROC), and some other Figures of Merit (FoM) have been used for the first time to evaluate and improve the performance of the prediction parameters and identify the optimum configuration to be used as a potential proxy of earthquake occurrence. As a result, a small, but detectable positive S4 anomaly was detected between 1 and 7 days before the earthquakes under study.

show abstract

“…The question of how to store and analyze this huge volume of data has been a popular field of research over the past decade, with various highly scalable computer architectures proposed in the literature (Zhao et al 2022). These architectures solve the volume problem by distributing processing over clusters of high-performance compute nodes (Sedona et al 2019) using parallel processing computing paradigms such as Hadoop/MapReduce (Boudriki Semlali and Freitag 2021;Rajak et al 2015;Tho et al 2020), Spark ("Apache Sedona," 2022;Ge et al 2019), Data Cubes (Appel and Pebesma 2019;"Open Data Cube," 2022;Simoes et al 2021), and scalable array databases (Câmara et al 2016;Cudre-Mauroux 2018;Joshi et al 2019). The most popular of these is the Google Earth Engine (Gorelick et al 2017) which is based around a parallel processing Hadoop/MapReduce architecture.…”

Section: Introductionmentioning

confidence: 99%

Two-Speed Deep-Learning Ensemble for Classification of Incremental Land-Cover Satellite Image Patches

et al. 2023

View full text Add to dashboard Cite

High-velocity data streams present a challenge to deep learning-based computer vision models due to the resources needed to retrain for new incremental data. This study presents a novel staggered training approach using an ensemble model comprising the following: (i) a resource-intensive high-accuracy vision transformer; and (ii) a fast training, but less accurate, low parameter-count convolutional neural network. The vision transformer provides a scalable and accurate base model. A convolutional neural network (CNN) quickly incorporates new data into the ensemble model. Incremental data are simulated by dividing the very large So2Sat LCZ42 satellite image dataset into four intervals. The CNN is trained every interval and the vision transformer trained every half interval. We call this combination of a complementary ensemble with staggered training a “two-speed” network. The novelty of this approach is in the use of a staggered training schedule that allows the ensemble model to efficiently incorporate new data by retraining the high-speed CNN in advance of the resource-intensive vision transformer, thereby allowing for stable continuous improvement of the ensemble. Additionally, the ensemble models for each data increment out-perform each of the component models, with best accuracy of 65% against a holdout test partition of the RGB version of the So2Sat dataset.

show abstract

SAT-Hadoop-Processor: A Distributed Remote Sensing Big Data Processing Software for Earth Observation Applications

Cited by 11 publications

References 35 publications

First Results on the Systematic Search of Land Surface Temperature Anomalies as Earthquakes Precursors

First Results on the Systematic Search of Land Surface Temperature Anomalies as Earthquakes Precursors

On the Correlation Between Earthquakes and Prior Ionospheric Scintillations Over the Ocean: A Study Using GNSS-R Data Between 2017 and 2021

Two-Speed Deep-Learning Ensemble for Classification of Incremental Land-Cover Satellite Image Patches

Contact Info

Product

Resources

About