Exascale computing and big data

Reed, Daniel A.; Dongarra, Jack

doi:10.1145/2699414

Cited by 360 publications

(165 citation statements)

References 17 publications

Supporting

Mentioning

154

Contrasting

Unclassified

Order By: Relevance

“…As the scientific community is preparing for the next-generation extreme-scale computing, big data analytics is becoming an essential part of the scientific process for insights and discoveries [25]. Effectively integrating big data analytics into HPC simulations still remains a daunting challenge, and it requires new workflow technologies to reduce user burden and improve efficiency.…”

Section: Hybrid Workflowsmentioning

confidence: 99%

See 1 more Smart Citation

A characterization of workflow management systems for extreme-scale applications

Silva

Filgueira

Pietri

et al. 2017

Future Generation Computer Systems

121

View full text Add to dashboard Cite

Automation of the execution of computational tasks is at the heart of improving scientific productivity. Over the last years, scientific workflows have been established as an important abstraction that captures data processing and computation of large and complex scientific applications. By allowing scientists to model and express entire data processing steps and their dependencies, workflow management systems relieve scientists from the details of an application and manage its execution on a computational infrastructure. As the resource requirements of today's computational and data science applications that process vast amounts of data keep increasing, there is a compelling case for a new generation of advances in high-performance computing, commonly termed as extreme-scale computing, which will bring forth multiple challenges for the design of workflow applications and management systems. This paper presents a novel characterization of workflow management systems using features commonly associated with extreme-scale computing applications. We classify 15 popular workflow management systems in terms of workflow execution models, heterogeneous computing environments, and data access methods. The paper also surveys workflow applications and identifies gaps for future research on the road to extreme-scale workflows and management systems.

show abstract

Section: Hybrid Workflowsmentioning

confidence: 99%

“…In addition, as the scientific community prepares for extreme-scale computing, big data analytics is becoming an essential part of the scientific process for insights and discoveries [25]. As big data applications became mainstream in recent years, new systems have been developed to handle the data processing.…”

Section: Introductionmentioning

confidence: 99%

A characterization of workflow management systems for extreme-scale applications

Silva

Filgueira

Pietri

et al. 2017

Future Generation Computer Systems

121

View full text Add to dashboard Cite

show abstract

“…The two order codes are then converted to the same value. If it is subtraction, the size of the two order codes needs to be compared [5]. After determining the symbol, the absolute value ΔE of the difference in the order code of the two operands is obtained.…”

Section: Floating Point Addition Unitmentioning

confidence: 99%

Design of Communication Oriented Kernel Components

2017

IJOMAM

View full text Add to dashboard Cite

Abstract:In order to develop a new algorithm used in core component design field, a research of processor core components based on communication algorithm is designed. The effectiveness of this kernel component is tested by three research methods. With the rapid growth of user requirements and application complexity, the mobile communications industry has undergone enormous changes in the base-band processor for communication algorithms applications. The performance of processor core functional components is an important factor that restricts the development of communications. The delay of functional components also affects the communication time of the base-band processor. Thus, the design and implementation of components have important theoretical and practical significance. The experimental results show that the floating-point unit based on communication algorithm is relatively low. Meanwhile, the design of processor oriented floating-point unit for communication algorithms is relatively simple, and the implementation conditions of these instructions are optimized from varying degrees. Based on the above finding, it is concluded that the communication algorithm is of great significance to the design of the internal core components of various systems.

show abstract

“…Creating checkpoints for many processes puts high pressure on the storage backend and on the network. Furthermore, the energy costs of moving data will exceed the costs of local computation in exascale environments [7]. A possible solution to increase checkpointing scalability is to apply data deduplication.…”

Section: Introductionmentioning

confidence: 99%