Science on the TeraGrid

Katz, Daniel S.; Callaghan, S.; Harkness, Robert; Jha, Shantenu; Kurowski, Krzysztof; Manos, Steven; Pamidighantam, Sudhakar; Pierce, Marlon; Plale, Beth; Song, Carol X.; Towns, John

doi:10.12921/cmst.2010.si.01.81-97

Cited by 5 publications

(4 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To assess scalability of our framework and test its performance in real-life applications, we performed a set of experiments using the TACC Ranger cluster [60]. A single node of this machine is based on AMD processors working at 2.3 GHz and provides 16 cores with 32 GB of DDR2 RAM and 512 KB of L2 cache per core.…”

Section: Resultsmentioning

confidence: 99%

Parallel Framework for Dimensionality Reduction of Large-Scale Datasets

Samudrala

Żola

Aluru

et al. 2015

Scientific Programming

View full text Add to dashboard Cite

Dimensionality reduction refers to a set of mathematical techniques used to reduce complexity of the original high-dimensional data, while preserving its selected properties. Improvements in simulation strategies and experimental data collection methods are resulting in a deluge of heterogeneous and high-dimensional data, which often makes dimensionality reduction the only viable way to gain qualitative and quantitative understanding of the data. However, existing dimensionality reduction software often does not scale to datasets arising in real-life applications, which may consist of thousands of points with millions of dimensions. In this paper, we propose a parallel framework for dimensionality reduction of large-scale data. We identify key components underlying the spectral dimensionality reduction techniques, and propose their efficient parallel implementation. We show that the resulting framework can be used to process datasets consisting of millions of points when executed on a 16,000-core cluster, which is beyond the reach of currently available methods. To further demonstrate applicability of our framework we perform dimensionality reduction of 75,000 images representing morphology evolution during manufacturing of organic solar cells in order to identify how processing parameters affect morphology evolution.

show abstract

Section: Resultsmentioning

confidence: 99%

Parallel Framework for Dimensionality Reduction of Large-Scale Datasets

Samudrala

Żola

Aluru

et al. 2015

Scientific Programming

View full text Add to dashboard Cite

show abstract

“…Similarly, ELIXIR's support for multiple authentication protocols is limited, hence it does not provide end user authentication with endentity X.509 certificates and LDAP based credentials. XSEDE [52] is the successor to TeraGrid [36], an NSF funded HPC and grid infrastructure. It consists of a collection of advanced digital resources and services (like supercomputers, visualization and storage systems, collections of data, software, networks, and expert support) that support researchers in various scientific domains.…”

Section: Related Workmentioning

confidence: 99%

“…The established e-infrastructures have been very successful, having provided resources to researchers on a national or multinational scale in TeraGrid [36], European National Grid Initiatives (NGIs), Extreme Science and Engineering Discovery Environments (XSEDEs) [52], or, in the case of the world-wide Large Hadron Colliders (LHCs) Computing Grid, a truly global scale [45]. They have provided data and compute resources in support of a vast range of research.…”

mentioning

confidence: 99%

Towards Federated Service Discovery and Identity Management in Collaborative Data and Compute Cloud Infrastructures

Memon

Jens

Willem³

et al. 2018

J Grid Computing

View full text Add to dashboard Cite

This paper compares three multinational research infrastructures, one that provides data services, one that provides compute services, and one that supports linguistics research. The aim is to jointly provide services to the user communities, and, perhaps eventually, seamlessly interoperate. To this end, we look at and compare how the infrastructures build their service federations (trust, service status, information systems), and how they manage users (identities, authentication, and authorisation).

show abstract

“…The TeraGrid [14,20] environment, which has now been Figure 2: Overview of the post-processing workflow for CyberShake. Light gray colored stages are run on XSEDE while the dark gray colored stage (seispeak) is run partially on OSG and XSEDE.…”

Section: Cyberinfrastructuresmentioning

confidence: 99%

Job and data clustering for aggregate use of multiple production cyberinfrastructures

Maheshwari

Espinosa

Wilde

et al. 2012

Proceedings of the Fifth International Workshop on Data-Intensive Distributed Computing Date

Self Cite

View full text Add to dashboard Cite

In this paper, we address the challenges of reducing the timeto-solution of the data intensive earthquake simulation workflow "CyberShake" by supplementing the high-performance parallel computing (HPC) resources on which it typically runs with distributed, heterogeneous resources that can be obtained opportunistically from grids and clouds. We seek to minimize time to solution by maximizing the amount of work that can be efficiently done on the distributed resources. We identify data movement as the main bottleneck in effectively utilizing the combined local and distributed resources. We address this by analyzing the I/O characteristics of the application, processor acquisition rate (from a pilot-job service), and the data movement throughput of the infrastructure. With these factors in mind, we explore a combination of strategies including partitioning of computation (over HPC and distributed resources) and job clustering.We validate our approach with a theoretical study and with preliminary measurements on the Ranger HPC system and distributed Open Science Grid resources. More complete performance results will be presented in the final submission of this paper.

show abstract

Science on the TeraGrid

Cited by 5 publications

References 26 publications

Parallel Framework for Dimensionality Reduction of Large-Scale Datasets

Parallel Framework for Dimensionality Reduction of Large-Scale Datasets

Towards Federated Service Discovery and Identity Management in Collaborative Data and Compute Cloud Infrastructures

Job and data clustering for aggregate use of multiple production cyberinfrastructures

Contact Info

Product

Resources

About