Many experiments have a data-management problem! This project aims at improving the situation, with adapted infrastructure and services Data Intensive Computing workflows are critical for the value of the data We present a concrete data intensive analysis use case at the LSDF 2 ICDIM 2011 Ariel García-LSDF Introduction The facility Software DIC Outlook What is the data challenge? Experiments getting exponentially more data ⇒ experiments need storage ⇒ need sophisticated data analysis workflows Science produces data! Old data is very valuable to analyse change in time for reprocessing for analysis by other scientists, in other contexts Invisible (not-found, no-metadata) data is lost data ⇒ adequate meta data greatly increases data value ⇒ single big scientific DB is more valuable than many small ones Data is used by large virtual communities! ⇒ access to data is critical
To address the growing need for value-added storage services for data intensive experiments at the Karlsruhe Institute of Technology (KIT), the Large Scale Data Facility (LSDF) was conceived and launched end of 2009. The LSDF main focus is to support scientific experiments producing large data sets reaching into the petabyte range in their data requirements. Working closely together with the involved scientific communities, the LSDF provides them not only with adequate storage space but also with an analysis facility and -more importantly-with support and value added services for data management, processing and preservation. In this poster we describe the approach taken to perform data analysis in LSDF, as well as for data management of the scientific datasets.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.