Computing the drainage network on huge grid terrains

Gomes, Thiago L.; Magalhães, Salles V. G.; Andrade, Marcus V. A.; Franklin, W. Randolph; Pena, Guilherme C.

doi:10.1145/2447481.2447488

Cited by 6 publications

(8 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…EMFlow running with a maximum of 2 GB RAM and tiles of 400 x 400 cells (the settings discussed by Gomes et al [12]) had 494 s wall-time and used 1.8 GB RAM. The new algorithm running with one consumer and 400 x 400 tiles had 1,015 s wall-time (2 x more) and used 674 MB RAM (2.7 x less).…”

Section: Resultsmentioning

confidence: 99%

Parallel Priority-Flood Depression Filling For Trillion Cell Digital Elevation Models On Desktops Or Clusters

Barnes

2016

Preprint

View full text Add to dashboard Cite

Algorithms for extracting hydrologic features and properties from digital elevation models (DEMs) are challenged by large datasets, which often cannot fit within a computer's RAM. Depression filling is an important preconditioning step to many of these algorithms. Here, I present a new, linearly-scaling algorithm which parallelizes the Priority-Flood depression-filling algorithm by subdividing a DEM into tiles. Using a single-producer, multi-consumer design, the new algorithm works equally well on one core, multiple cores, or multiple machines and can take advantage of large memories or cope with small ones. Unlike previous algorithms, the new algorithm guarantees a fixed number of memory access and communication events per subdivision of the DEM. In comparison testing, this results in the new algorithm running generally faster while using fewer resources than previous algorithms. For moderately sized tiles, the algorithm exhibits ∼ 60% strong and weak scaling efficiencies up to 48 cores, and linear time scaling across datasets ranging over three orders of magnitude. The largest dataset on which I run the algorithm has 2 trillion (2 • 10 12 ) cells. With 48 cores, processing required 4.8 hours wall-time (9.3 compute-days). This test is three orders of magnitude larger than any previously performed in the literature. Complete, well-commented source code and correctness tests are available for download from a repository.

show abstract

Section: Resultsmentioning

confidence: 99%

Parallel Priority-Flood Depression Filling For Trillion Cell Digital Elevation Models On Desktops Or Clusters

Barnes

2016

Preprint

View full text Add to dashboard Cite

show abstract

“…Although using integer data limits the scope of their method, they claim it is a significant improvement over the floating-point algorithms presented by Wang and Liu [2006] and Liu et al [2009]. Gomes et al [2012] extends the work of Magalhães et al [2012] to increase the efficiency of the Priority-Flood Algorithm in situations where memory access must be limited.…”

Section: Historymentioning

confidence: 99%

Priority-flood: An optimal depression-filling and watershed-labeling algorithm for digital elevation models

Barnes

Lehman

Mulla

2014

Computers & Geosciences

213

227

View full text Add to dashboard Cite

Depressions (or pits) are low areas within a digital elevation model that are surrounded by higher terrain, with no outlet to lower areas. Filling them so they are level, as fluid would fill them if the terrain were impermeable, is often necessary in preprocessing DEMs. The depression-filling algorithm presented herecalled Priority-Flood-unifies and improves on the work of a number of previous authors who have published similar algorithms. The algorithm operates by flooding DEMs inwards from their edges using a priority queue to determine the next cell to be flooded. The resultant DEM has no depressions or digital dams: every cell is guaranteed to drain. The algorithm is optimal for both integer and floating-point data, working in O(n) and O(n log 2 n) time, respectively. It is shown that by using a plain queue to fill depressions once they have been found, an O(m log 2 m) time-complexity can be achieved, where m does not exceed the number of cells n. This is the lowest time complexity of any known floating-point depression-filling algorithm. In testing, this improved variation of the algorithm performed up to 37% faster than the original. Additionally, a parallel version of an older, but widely-used depression-filling algorithm required six parallel processors to achieve a run-time on par with what the newer algorithm's improved variation took on a single processor. The Priority-Flood Algorithm is simple to understand and implement: the included pseudocode is only 20 lines and the included C++ reference implementation is under a hundred lines. The algorithm can work on irregular meshes as well as 4-, 6-, 8-, and n-connected grids. It can also be adapted to label watersheds and determine flow directions through either incremental elevation changes or depression carving. In the case of incremental elevation changes, the algorithm includes safety checks not present in prior works.

show abstract

“…In a fourth test, a comparison was made against the work of both Wallis et al [2009] (TauDEM 5 ) and Gomes et al [2012] (EMFlow 6 ). These algorithms have source code available, claim to be suitable for large datasets, and claim to be faster than other algorithms, including ArcGIS.…”

Section: Empirical Testsmentioning

confidence: 99%

“…Existing algorithms [Arge et al, 2003;Danner et al, 2007;Do et al, 2011;Gomes et al, 2012;Lindsay, 2015;Metz et al, 2010Metz et al, , 2011Tesfa et al, 2011;Wallis et al, 2009;Yao and Shi, 2015;Yıldırım et al, 2015] have taken one of two approaches to DEMs that cannot fit entirely into RAM. They either (a) keep only a subset of the DEM in RAM at any time by using virtual tiles stored to a computer's hard disk or (b) keep the entire DEM in RAM by distributing it over multiple compute nodes which communicate with each other.…”

Section: Introductionmentioning

confidence: 99%

Parallel non-divergent flow accumulation for trillion cell digital elevation models on desktops or clusters

Barnes¹

2017

Environmental Modelling & Software

View full text Add to dashboard Cite

Continent-scale datasets challenge hydrological algorithms for processing digital elevation models. Flow accumulation is an important input for many such algorithms; here, I parallelize its calculation. The new algorithm works on one or many cores, or multiple machines, and can take advantage of large memories or cope with small ones. Unlike previous parallel algorithms, the new algorithm guarantees a fixed number of memory access and communication events per raster cell. In testing, the new algorithm ran faster and used fewer resources than previous algorithms, exhibiting ∼30% strong and weak scaling efficiencies up to 48 cores and linear scaling across datasets ranging over three orders of magnitude. The largest dataset tested had two trillion (2 · 10 12 ) cells. With 48 cores, processing required 24 minutes wall-time (14.5 compute-hours).This test is three orders of magnitude larger than any previously performed in the literature. Complete, well-commented source code and correctness tests are available on Github.Keywords: parallel computing, hydrology, geographic information system (GIS), upslope area, contributing area SoftwareComplete, well-commented source code, an associated makefile, and correctness tests are available at https: //github.com/r-barnes/Barnes2016-ParallelFlowAccum. The code is written in C++ using MPI and constitutes 2,131 lines of code of which 58% are or contain comments.This algorithm is part of the RichDEM (https://github.com/r-barnes/richdem) terrain analysis suite, a collection of state of the art algorithms for processing large digital elevation models quickly.

show abstract

Computing the drainage network on huge grid terrains

Cited by 6 publications

References 9 publications

Parallel Priority-Flood Depression Filling For Trillion Cell Digital Elevation Models On Desktops Or Clusters

Parallel Priority-Flood Depression Filling For Trillion Cell Digital Elevation Models On Desktops Or Clusters

Priority-flood: An optimal depression-filling and watershed-labeling algorithm for digital elevation models

Parallel non-divergent flow accumulation for trillion cell digital elevation models on desktops or clusters

Contact Info

Product

Resources

About