Schema-Independent and Schema-Friendly Scientific Metadata Management

Jensen, Scott A.; Plale, Beth

doi:10.1109/escience.2008.114

Cited by 3 publications

(5 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These data collections, accessible through the Purdue data management system 4 A large number of satellite data products can be generated from the raw satellite observations. For instance, data from the MODIS sensor on the Terra and Aqua satellites can generate 14 different products.…”

Section: Iv2 Realtime Satellite Data Gadgetsmentioning

confidence: 99%

“…These data collections, accessible through the Purdue data management system 4 , are linked with processing and visualization capabilities provided by TeraGrid resources. Among the observational data, Purdue makes available the remote sensing satellite data from the Purdue Terrestrial Observa-4 http://www.purdue.teragrid.org/portal tory's 5 two satellite ground stations, which are capable of continuous ingestion of remote sensing data from the NOAA GOES-12, NASA MODIS Terra/Aqua, NOAA AVHRR, and Fengyun-1D MVISR sensor systems.…”

Section: Iv2 Realtime Satellite Data Gadgetsmentioning

confidence: 99%

“…Model-generated radar results of a storm that caused severe tornado damage in Norman, Oklahoma. The forecast is generated using ADAS 5.10, WRF 3.1.1, and NCL to generate the visualizations, and runs on the LEAD II gateway LEAD II pioneered XBaya workflow composer [3], XMC Cat metadata catalog [4], and the GFAC service wrapper [5]. The workflows are computationally intense and take several hours to run the "typical workflow" that has forecast duration of greater than 36 hours, and then ingest and produce gigabyte-sized data files.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Science on the TeraGrid

Katz¹,

Callaghan²,

Harkness³

et al. 2010

CMST

Self Cite

View full text Add to dashboard Cite

Abstract:The TeraGrid is an advanced, integrated, nationally-distributed, open, user-driven, US cyberinfrastructure that enables and supports leading edge scientific discovery and promotes science and technology education. It comprises supercomputing resources, storage systems, visualization resources, data collections, software, and science gateways, integrated by software systems and high bandwidth networks, coordinated through common policies and operations, and supported by technology experts. This paper discusses the TeraGrid itself, examples of the science that is occurring on the TeraGrid today, and applications that are being developed to perform science in the future. Key words: computational science applications, high performance computing, grid computing, production grid infrastructure *Steven Manos was a Research Fellow at University College London while this work was conducted CMST SI(1) 81-97 (2010 82effort to build and deploy the world's largest, fastest, most comprehensive, distributed infrastructure for general scientific research. The initial TeraGrid was homogeneous and very "griddy", with users foreseen to be running on multiple systems, both because their codes could run "anywhere", and because in some cases, multiple systems would be needed to support the large runs that were desired. The software that made up the TeraGrid was a set of identical packages on all systems. The TeraGrid has since expanded in capability and number of resource providers. This introduced heterogeneity and thus added complexity to the grid ideals of the initial DTF, as the common software no longer could be completely identical. This led to the concept of common interfaces, with potentially different software underneath the interfaces. Additionally, the users of national center supercomputers were merged into TeraGrid, which led to TeraGrid increasing its focus on supporting these users and their traditional parallel/batch usage modes. The TeraGrid is freely available for US researchers, with allocations determined by a peer-review mechanism. The TeraGrid resources are generally at least 50% dedicated to TeraGrid. Roughly similar to the TeraGrid in terms of a continental-scale set of resources with a focus on HPC users is DEISA/PRACE in Europe, which is a set of national resources with some fractions provided to a set of European users, again determined through a review process. An alternative type of infrastructure is one that is more focused on high throughput computing (HTC), initially in high-energy physics, but increasingly in other domains as well. Filling this role in the US is Open Science Grid (OSG), and in Europe, Enabling Grids for E-sciencE (EGEE), which has recently been transformed into the European Grid Infrastructure (EGI). This paper discusses the TeraGrid itself (Section II), examples of the interesting current uses of the TeraGrid cyberinfrastructure for science (Section III), and examples of applications that are being developed to perform science on the TeraGrid in the future (Section IV) 1 . ...

show abstract

Section: Iv2 Realtime Satellite Data Gadgetsmentioning

confidence: 99%

Section: Iv2 Realtime Satellite Data Gadgetsmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Science on the TeraGrid

Katz¹,

Callaghan²,

Harkness³

et al. 2010

CMST

Self Cite

View full text Add to dashboard Cite

show abstract

“…It pioneered the community account as a way for researchers to execute computationally intensive analysis and modeling tools on TeraGrid resources. Users are provided private data storage for data they create in the gateway, and the data are organized through a metadata catalog [20].…”

Section: Linked Environments For Atmospheric Discoverymentioning

confidence: 99%

What is cyberinfrastructure

Stewart

Simms

Plale

et al. 2010

Proceedings of the 38th Annual ACM SIGUCCS Fall Conference: Navigation and Discovery

Self Cite

View full text Add to dashboard Cite

Cyberinfrastructure is a word commonly used but lacking a single, precise definition. One recognizes intuitively the analogy with infrastructure, and the use of cyber to refer to thinking or computing -but what exactly is cyberinfrastructure as opposed to information technology infrastructure? Indiana University has developed one of the more widely cited definitions of cyberinfrastructure:Cyberinfrastructure consists of computing systems, data storage systems, advanced instruments and data repositories, visualization environments, and people, all linked together by software and high performance networks to improve research productivity and enable breakthroughs not otherwise possible.A second definition, more inclusive of scholarship generally and educational activities, has also been published and is useful in describing cyberinfrastructure:Cyberinfrastructure consists of computational systems, data and information management, advanced instruments, visualization environments, and people, all linked together by software and advanced networks to improve scholarly productivity and enable knowledge breakthroughs and discoveries not otherwise possible.In this paper, we describe the origin of the term cyberinfrastructure based on the history of the root word infrastructure, discuss several terms related to cyberinfrastructure, and provide several examples of cyberinfrastructure.

show abstract

“…It adapts to community metadata schemata by partitioning each schema based on the metadata concepts it contains [20]. When ingesting metadata, character large objects (CLOBs) containing the XML for each concept are stored in the catalog and later used to rebuild the full metadata for an object in response to queries.…”

Section: B Introduction To Xmc Catmentioning

confidence: 99%

Trading Consistency for Scalability in Scientific Metadata

Jensen

Plale

2010

2010 IEEE Sixth International Conference on E-Science

Self Cite

View full text Add to dashboard Cite

Abstract-Long-term repositories that are able to represent the detailed descriptive metadata of scientific data have been recognized as key to both data reuse and preservation of the initial investment in generating the data. Detailed metadata captured during scientific investigation not only enables the efficient discovery of relevant data sets but also is a source for exploring ongoing activity. In XMC Cat metadata catalog, an XML catalog that uses a novel hybrid model to store XML to a relational database, we exploit differences in the temporal utility between browse and search metadata to selectively relax the consistency model used. By ensuring only eventual consistency on parts of the solution, we determine through experimental analysis that the performance and scalability of the catalog can be substantially improved.

show abstract

Schema-Independent and Schema-Friendly Scientific Metadata Management

Cited by 3 publications

References 11 publications

Science on the TeraGrid

Science on the TeraGrid

What is cyberinfrastructure

Trading Consistency for Scalability in Scientific Metadata

Contact Info

Product

Resources

About