Geographical information system parallelization for spatial big data processing: a review

Zhao, Lingjun; Chen, Lajiao; Ranjan, Rajiv; Choo, Kim‐Kwang Raymond; He, Jijun

doi:10.1007/s10586-015-0512-2

Cited by 60 publications

(36 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, as compared with CPU clusters, GPU clusters have more expensive hardware and the associativity between software and hardware requires improvement [23]. Therefore, considering a series of factors, most geo-computing parallel algorithms use the HPC with multiple CPUs as the parallel environment to achieve a variety of characteristics of GIS computing tasks [11,24,25]. Thus, the aim of this research was to design a high-performance parallel algorithm for generating DEM from LiDAR points in the HPC with multiple CPUs.…”

Section: Related Workmentioning

confidence: 99%

“…Thus, using parallel computing to improve the processing efficiency of spatial data has been a hot topic in the field of geo-computing. A series of parallel geo-computing algorithms has been proposed, such as spatial data conversion algorithm [8], remote sensing images processing algorithm [9], and raster computing algorithms [10][11][12], etc. This provides important references for LiDAR dataset processing.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Hybrid Process/Thread Parallel Algorithm for Generating DEM from LiDAR Points

Ren

Chen

et al. 2017

IJGI

View full text Add to dashboard Cite

Airborne Light Detection and Ranging (LiDAR) is widely used in digital elevation model (DEM) generation. However, the very large volume of LiDAR datasets brings a great challenge for the traditional serial algorithm. Using parallel computing to accelerate the efficiency of DEM generation from LiDAR points has been a hot topic in parallel geo-computing. Generally, most of the existing parallel algorithms running on high-performance clusters (HPC) were in process-paralleling mode, with a static scheduling strategy. The static strategy would not respond dynamically according to the computation progress, leading to load unbalancing. Additionally, because each process has independent memory space, the cost of dealing with boundary problems increases obviously with the increase in the number of processes. Actually, these two problems can have a significant influence on the efficiency of DEM generation for larger datasets, especially for those of irregular shapes. Thus, to solve these problems, we combined the advantages of process-paralleling with the advantages of thread-paralleling, forming a new idea: using process-paralleling to achieve a flexible schedule and scalable computation, using thread-paralleling inside the process to reduce boundary problems. Therefore, we proposed a hybrid process/thread parallel algorithm for generating DEM from LiDAR points. Firstly, at the process level, we designed a parallel method (PPDB) to accelerate the partitioning of LiDAR points. We also proposed a new dynamic scheduling strategy to achieve better load balancing. Secondly, at the thread level, we designed an asynchronous parallel strategy to hide the cost of LiDAR points' reading. Lastly, we tested our algorithm with three LiDAR datasets. Experiments showed that our parallel algorithm had no influence on the accuracy of the resultant DEM. At the same time, our algorithm reduced the conversion time from 112,486 s to 2342 s when we used the largest dataset (150 GB). The PPDB was parallelizable and the new dynamic scheduling strategy achieved a better load balancing. Furthermore, the asynchronous parallel strategy reduced the impact of LiDAR points reading. When compared with the traditional process-paralleling algorithm, the hybrid process/thread parallel algorithm improved the conversion efficiency by 30%.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Hybrid Process/Thread Parallel Algorithm for Generating DEM from LiDAR Points

Ren

Chen

et al. 2017

IJGI

View full text Add to dashboard Cite

show abstract

“…There were also concerns about Cloud Service providers (CSPs) being able to infer or profile users based on the hosted/stored data [3], [4], [5]. Thus, it is unsurprising that cloud and big data security and privacy are current research focus [6], [7], [8], [9], [10], [11], [12], [13], [14]. Data owners in industries such as healthcare and banking are also subject to exacting regulatory requirements, which restrict the outsourcing of the data for storage and analysis.…”

Section: Introductionmentioning

confidence: 99%

Greening Cloud-Enabled Big Data Storage Forensics: Syncany as a Case Study

Teing

Dehghantanha

Choo

et al. 2019

IEEE Trans. Sustain. Comput.

Self Cite

View full text Add to dashboard Cite

Abstract-The pervasive nature of cloud-enabled big data storage solutions introduces new challenges in the identification, collection, analysis, preservation and archiving of digital evidences. Investigation of such complex platforms to locate and recover traces of criminal activities is a time-consuming process. Hence, cyber forensics researchers are moving towards streamlining the investigation process by locating and documenting residual artefacts (evidences) of forensic value of users' activities on cloud-enabled big data platforms in order to reduce the investigation time and resources involved in a real-world investigation. In this paper, we seek to determine the data remnants of forensic value from Syncany private cloud storage service, a popular storage engine for big data platforms. We demonstrate the types and the locations of the artefacts that can be forensically recovered. Findings from this research contribute to an in-depth understanding of cloud-enabled big data storage forensics, which can result in reduced time and resources spent in real-world investigations involving Syncany-based cloud platforms.

show abstract

“…Due to the increasing amount of data that needs to be processed [10][11][12][13], feature selection can be used in the pre-processing phase before classification in order to identify important features of a dataset, with the aims of improving prediction accuracy and reducing computational complexity. Existing defence methods that are capable of handling significant amount of data generally contain redundant or irrelevant features, which result in excessive training and classification time [14].…”

Section: Introductionmentioning

confidence: 99%

Ensemble-based multi-filter feature selection method for DDoS detection in cloud computing

Osanaiye

Cai

Choo

et al. 2016

J Wireless Com Network

Self Cite

263

136

View full text Add to dashboard Cite

Widespread adoption of cloud computing has increased the attractiveness of such services to cybercriminals. Distributed denial of service (DDoS) attacks targeting the cloud's bandwidth, services and resources to render the cloud unavailable to both cloud providers, and users are a common form of attacks. In recent times, feature selection has been identified as a pre-processing phase in cloud DDoS attack defence which can potentially increase classification accuracy and reduce computational complexity by identifying important features from the original dataset during supervised learning. In this work, we propose an ensemble-based multi-filter feature selection method that combines the output of four filter methods to achieve an optimum selection. We then perform an extensive experimental evaluation of our proposed method using intrusion detection benchmark dataset, NSL-KDD and decision tree classifier. The findings show that our proposed method can effectively reduce the number of features from 41 to 13 and has a high detection rate and classification accuracy when compared to other classification techniques.

show abstract

Geographical information system parallelization for spatial big data processing: a review

Cited by 60 publications

References 60 publications

A Hybrid Process/Thread Parallel Algorithm for Generating DEM from LiDAR Points

A Hybrid Process/Thread Parallel Algorithm for Generating DEM from LiDAR Points

Greening Cloud-Enabled Big Data Storage Forensics: Syncany as a Case Study

Ensemble-based multi-filter feature selection method for DDoS detection in cloud computing

Contact Info

Product

Resources

About