A Codesign Framework for Online Data Analysis and Reduction

Mehta, Kshitij; Foster, Ian; Klasky, Scott; Allen, Bryce; Wolf, Matthew; Logan, Jeremy; Suchyta, E.; Choi, Jong Youl; Takahashi, Keichi; Yakushin, I.; Munson, Todd

doi:10.1109/works49585.2019.00007

Cited by 14 publications

(13 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such MPMD computations have implications for both programming models and implementations. These needs motivated CODAR work on both MPI extensions—the MPI_Comm_launch of Wozniak et al (2019)—and libraries for creating and coupling subcomputations (Mehta et al, 2019).…”

Section: Co-design Technologiesmentioning

confidence: 99%

See 1 more Smart Citation

Online data analysis and reduction: An important Co-design motif for extreme-scale computers

Foster

Ainsworth

Bessac

et al. 2021

The International Journal of High Performance Computing Applica

Self Cite

View full text Add to dashboard Cite

A growing disparity between supercomputer computation speeds and I/O rates means that it is rapidly becoming infeasible to analyze supercomputer application output only after that output has been written to a file system. Instead, data-generating applications must run concurrently with data reduction and/or analysis operations, with which they exchange information via high-speed methods such as interprocess communications. The resulting parallel computing motif, online data analysis and reduction (ODAR), has important implications for both application and HPC systems design. Here we introduce the ODAR motif and its co-design concerns, describe a co-design process for identifying and addressing those concerns, present tools that assist in the co-design process, and present case studies to illustrate the use of the process and tools in practical settings.

show abstract

Section: Co-design Technologiesmentioning

confidence: 99%

“…Cheetah supports the specification of a campaign; Savanna executes a campaign on a target system. From Mehta et al (2019).…”

Section: Co-design Technologiesmentioning

confidence: 99%

Online data analysis and reduction: An important Co-design motif for extreme-scale computers

Foster

Ainsworth

Bessac

et al. 2021

The International Journal of High Performance Computing Applica

Self Cite

View full text Add to dashboard Cite

show abstract

“…The work with DeepDriveMD was already mentioned, as well as online data analysis that enables large speedups on a code that won a 2020 Gordon Bell award. Publications detail investigations of compression quality trade-offs [61], work with Cheetah [62], and work with EFFIS [63]. Work with Chimbuko that applied NWChem on Summit won the best paper award at the 2020 In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization conference [64].…”

Section: Data Reductionmentioning

confidence: 99%

Map Applications to Target Exascale Architecture with Machine-Specific Performance Analysis, Including Challenges and Projections

Siegel

Evans

Draeger

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Co-design allows integrated design and testing of the rapidly developing software stack with emerging hardware technologies while developing software components that embody the most common application motifs [4]. Specific co-design exercises are called campaigns: a multi-researcher co-design activity that contributes to a single scientific development objective [10].…”

Section: Motivationmentioning

confidence: 99%

Campaign Knowledge Network: Building Knowledge for Campaign Efficiency

Withana¹,

Mehta²,

Wolf³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

In the landscape of exascale computing collaborative research campaigns are conducted as co-design activities of loosely coordinated experiments. But the higher level context and the knowledge of individual experimental activity is lost over time. We undertook a knowledge capture and representation aid called Campaign Knowledge Network(CKN), a codesign design and analysis tool. We demonstrate that CKN can satisfy the Hoarde abstraction and can distill campaign context from runtime information thereby creating a knowledge resource upon which analysis tools can run to provide more efficient experimentation.

show abstract

A Codesign Framework for Online Data Analysis and Reduction

Cited by 14 publications

References 18 publications

Online data analysis and reduction: An important Co-design motif for extreme-scale computers

Online data analysis and reduction: An important Co-design motif for extreme-scale computers

Map Applications to Target Exascale Architecture with Machine-Specific Performance Analysis, Including Challenges and Projections

Campaign Knowledge Network: Building Knowledge for Campaign Efficiency

Contact Info

Product

Resources

About