A comparative analysis of parallel disk-based Methods for enumerating implicit graphs

Robinson, Eric; Kunkle, Daniel; Cooperman, Gene

doi:10.1145/1278177.1278190

Cited by 5 publications

(5 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using the same approach for an out-of-core implementation is impractical, because depth-first search (DFS) relies heavily on random data access. It is well-known that streaming access should be used in a disk-based implementation instead of random access [25].…”

Section: A General Methods For Migrating Recursive Algorithms From Thementioning

confidence: 99%

“…An important part of many symbolic computations is working with very large graph-like data structures, that only fit on parallel disks: groups [25], binary decision diagrams [19] and others. Very large finite state automata are useful in many areas, within symbolic computations and outside of it, as discussed in Section 3.…”

Section: Introductionmentioning

confidence: 99%

“…When working with the above mentioned large graph-like data structures, techniques similar to dynamic programming are often employed (for example, see [19,25]). Migrating dynamic programming to parallel disks is described in Section 2.1 and in our parallel disk-based implementation of the 0-1 Knapsack problem in Section 4.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An efficient programming model for memory-intensive recursive algorithms using parallel disks

Slavici

Kunkle

Cooperman

et al. 2012

Proceedings of the 37th International Symposium on Symbolic and Algebraic Computation

Self Cite

View full text Add to dashboard Cite

In order to keep up with the demand for solutions to problems with ever-increasing data sets, both academia and industry have embraced commodity computer clusters with locally attached disks or SANs as an inexpensive alternative to supercomputers. With the advent of tools for parallel disks programming, such as MapReduce, STXXL and Roomythat allow the developer to focus on higher-level algorithms -the programmer productivity for memory-intensive programs has increased many-fold. However, such parallel tools were primarily targeted at iterative programs.We propose a programming model for migrating recursive RAM-based legacy algorithms to parallel disks. Many memory-intensive symbolic algebra algorithms are most easily expressed as recursive algorithms. In this case, the programming challenge is multiplied, since the developer must re-structure such an algorithm with two criteria in mind: converting a naturally recursive algorithm into an iterative algorithm, while simultaneously exposing any potential data parallelism (as needed for parallel disks).This model alleviates the large effort going into the design phase of an external memory algorithm. Research in this area over the past 10 years has focused on per-problem solutions, without providing much insight into the connection between legacy algorithms and out-of-core algorithms. Our method shows how legacy algorithms employing recursion and non-streaming memory access can be more easily translated into efficient parallel disk-based algorithms.We demonstrate the ideas on a largest computation of its kind: the determinization via subset construction and minimization of very large nondeterministic finite set automata (NFA). To our knowledge, this is the largest subset construction reported in the literature. Determinization for large NFA has long been a large computational hurdle in the study of permutation classes defined by token passing networks. The programming model was used to design and implement an efficient NFA determinization algorithm that solves the next stage in analyzing token passing networks representing two stacks in series.

show abstract

Section: A General Methods For Migrating Recursive Algorithms From Thementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An efficient programming model for memory-intensive recursive algorithms using parallel disks

Slavici

Kunkle

Cooperman

et al. 2012

Proceedings of the 37th International Symposium on Symbolic and Algebraic Computation

Self Cite

View full text Add to dashboard Cite

show abstract

“…For lower-level memory data, some of the main ideas of disk-based computing [14,16] have been used successfully in recent years to solve or make progress on important problems in computational group theory [9,10,14,15], where the size of the data is too large for one RAM subsystem or even the aggregate RAM of a cluster.…”

Section: Related Workmentioning

confidence: 99%

Fast multiplication of large permutations for disk, flash memory and RAM

Slavici

Dong

Kunkle

et al. 2010

Proceedings of the 2010 International Symposium on Symbolic and Algebraic Computation

Self Cite

View full text Add to dashboard Cite

Permutation multiplication (or permutation composition) is perhaps the simplest of all algorithms in computer science. Yet for large permutations, the standard algorithm is not the fastest for disk or for flash, and surprisingly, it is not even the fastest algorithm for RAM on recent multi-core CPUs. On a recent commodity eight-core machine we demonstrate a novel algorithm that is 50% faster than the traditional algorithm. For larger permutations on flash or disk, the novel algorithm is orders of magnitude faster. A disk-parallel algorithm is demonstrated that can multiply two permutations with 12.8 billion points using 16 parallel local disks of a cluster in under one hour. Such large permutations are important in computational group theory, where they arise as the result of the well-known Todd-Coxeter coset enumeration algorithm. The novel algorithm emphasizes several passes of streaming access to the data instead of the traditional single pass using random access to the data. Similar novel algorithms are presented for permutation inverse and permutation multiplication by an inverse, thus providing a complete library of the underlying permutation operations needed for computations with permutation groups.

show abstract

“…Our team is now searching for an organizing principle that will allow for the construction of a software library or language extension that does for disk-based computation what numerical libraries have done for numerical analysis. As an initial step, we have begun a comparative analysis of eight different techniques for disk-based enumeration [2]. This analysis is based on the methods we used for Rubik's Cube, along with our solutions to several model problems in computational group theory.…”

Section: Organizing Principlementioning

confidence: 99%