Performance characteristics of Graph500 on large-scale distributed environment

Suzumura, Toyotaro; Ueno, K.; Sato, Hitoshi; Matsuoka, Satoshi

doi:10.1109/iiswc.2011.6114175

Cited by 52 publications

(37 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We conducted a thorough study of the algorithms of the reference implementations and their performance in an earlier paper [32]. Based on that work, we implemented a scalable and high-performance implementation of an optimized Graph500 benchmark for large distributed environments [37] [38].…”

Section: Graph500 Benchmarkmentioning

confidence: 99%

See 1 more Smart Citation

Extreme Big Data (EBD): Next Generation Big Data Infrastructure Technologies Towards Yottabyte/Year

Matsuoka

Sato

Tatebe

et al. 2014

JSFI

View full text Add to dashboard Cite

Miyoshi 5Our claim is that so-called "Big Data" will evolve into a new era with proliferation of data from multiple sources such as massive numbers of sensors whose resolution is increasing exponentially, high-resolution simulations generating huge data results, as well as evolution of social infrastructures that allow for "opening up of data silos", i.e., data sources being abundant across the world instead of being confined within an institution, much as how scientific data are being handled in the modern era as a common asset openly accessible within and across disciplines. Such a situation would create the need for not only petabytes to zetabytes of capacity and beyond, but also for extreme scale computing power. Our new project, sponsored under the Japanese JST-CREST program is called "Extreme Big Data", and aims to achieve the convergence of extreme supercomputing and big data in order to cope with such explosion of data. The project consists of six teams, three of which deals with defining future EBD convergent SW/HW architecture and system, and the other three the EBD co-design applications that represent different facets of big data, in metagenomics, social simulation, and climate simulation with real-time data assimilation. Although the project is still early in its lifetime, started in Oct. 2013, we have already achieved several notable results, including becoming world #1 on the Green Graph 500, a benchmark to measure the power efficiency of graph processing that appear in typical big data scenarios.

show abstract

Section: Graph500 Benchmarkmentioning

confidence: 99%

“…For example, the level Scale 26 for toy means 226 and corresponds to 1010 bytes occupying 17 GB of memory. The six Scale values are 26,29,32,36,39, and 42 for the six classes. The largest problem, huge (Scale 42), needs to handle around 1.1 PB of memory.…”

Section: Graph500 Benchmarkmentioning

confidence: 99%

Extreme Big Data (EBD): Next Generation Big Data Infrastructure Technologies Towards Yottabyte/Year

Matsuoka

Sato

Tatebe

et al. 2014

JSFI

View full text Add to dashboard Cite

show abstract

“…Local discovery (per substep) (lines [11][12][13][14][15][16][17][18][19][20] Search for parents with the information available locally.…”

Section: Parallel and Distributed Bfs Algorithmmentioning

confidence: 99%

“…We have conducted a series of work [11][12][13] to accelerate BFS in a distributed-memory environment. Our new work extends the data structures and algorithm called hybrid BFS [2] that is known to be effective small-diameter graphs, so that it scales to top-tier supercomputers with tens of thousands of nodes with million-scale CPU cores with multi-gigabyte/s interconnect.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Efficient Breadth-First Search on Massively Parallel and Distributed-Memory Machines

et al. 2017

Self Cite

View full text Add to dashboard Cite

Applying search algorithms to obtain the optimal configuration of nDT torus nodes

Andújar‐Muñoz

Villar-Ortiz

Sánchez

et al. 2017

Concurrency and Computation

View full text Add to dashboard Cite

SummaryAn nDT torus is a topology where each node comprises 2 identical (n+1)‐port communication cards interconnected by 1 port. By using the current switches or communication cards, this node architecture allows to build torus networks having a greater number of dimensions than networks including only 1 card per node. There are multiple ways to use the ports of the 2 cards to connect a node to other nodes on the nDT torus, and therefore, checking all the configurations is only an affordable problem for small values of n. In this paper, we use artificial intelligence and data mining techniques to obtain the optimal port configuration of all the nodes in the network. We include a performance evaluation that shows nDT torus effectively increases the performance compared with the equivalent torus in resources, with synthetic and application trace–based workloads. We also apply these techniques to 3DT and 5DT tori to confirm the increase in the number of dimensions that does not affect to performance of the nDT torus.

show abstract

Performance characteristics of Graph500 on large-scale distributed environment

Cited by 52 publications

References 12 publications

Extreme Big Data (EBD): Next Generation Big Data Infrastructure Technologies Towards Yottabyte/Year

Extreme Big Data (EBD): Next Generation Big Data Infrastructure Technologies Towards Yottabyte/Year

Efficient Breadth-First Search on Massively Parallel and Distributed-Memory Machines

Applying search algorithms to obtain the optimal configuration of nDT torus nodes

Contact Info

Product

Resources

About