The CMS dataset bookkeeping service

Afaq, A.; Dolgert, A.; Guo, Yuyi; Jones, Chris D.; Kosyakov, S.; Kuznetsov, Valentin; Lueking, L.; Riley, Dan; Sekhri, V.

doi:10.1088/1742-6596/119/7/072001

Cited by 35 publications

(25 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Histograms are defined for each roll or for larger detector segments, such as the the sectors (60) and wheels (5) in the barrel and rings (12) and disks (6) in the forward regions (see section 1). Event selection is performed at this level.…”

Section: Rpc Dqm Structurementioning

confidence: 99%

See 1 more Smart Citation

Data Quality Monitoring for the CMS Resistive Plate Chamber Detector

Cimmino¹

2011

Nuclear Physics B - Proceedings Supplements

View full text Add to dashboard Cite

Resistive Plate Chambers (RPCs), with their excellent time resolution (∼ ns), were chosen as dedicated muon trigger detectors for the CMS experiment. RPCs fulfill the job of muon identification, estimate the momentum and unambiguously assign bunch crossing. The critical tasks of monitoring detector performance, debugging hardware, and certifying recorded data are carried out by the Data Quality Monitoring (DQM) system. The CMS DQM framework provides tools for creation, filling, storage, and visualization of histograms and scalar elements. It also offers standardized algorithms for performing statistical tests and automated data certification. Within this framework, the RPC DQM system was developed. The later is composed by a set of user defined algorithms and is intended to be used both online, during data taking, and offline, during the reconstruction stage at Tier-0 and re-reconstruction at the Tier-1s. Run by run, the system measures detector level and physics quantities which are subsequently stored in a dedicate database. We here describe the structure, functionalities, and performance of the DQM applications for the CMS RPC detector.

show abstract

Section: Rpc Dqm Structurementioning

confidence: 99%

“…Weekly meetings are held to collect the final decisions and communicate them to the entire CMS collaboration. At this stage quality flags are copied to the offline condition database and to the Dataset Bookkeeping System (DBS) [12]. During the year 2009, RPC DQM certified 1979 runs, 547 of which are in presence of beams.…”

Section: Data Certificationmentioning

confidence: 99%

Data Quality Monitoring for the CMS Resistive Plate Chamber Detector

Cimmino¹

2011

Nuclear Physics B - Proceedings Supplements

View full text Add to dashboard Cite

show abstract

“…The Dataset Bookkeeping Service (DBS) [3] provides the means to describe, discover and use CMS event data. It catalogs CMS specific data definitions such as run number, the algorithms and configurations used to process the data together with the information regarding the processing parentage of the data it describes.…”

Section: Dbsmentioning

confidence: 99%

Distributed Analysis in CMS

et al. 2010

View full text Add to dashboard Cite

CMS expects to manage several Pbytes of data each year, distributing them over many computing sites around the world and enabling data access at those centers for analysis. CMS has identified the distributed sites as the primary location for physics analysis to support a wide community of users, with potentially as many as 3000 users. This represents an unprecedented scale of distributed computing resources and number of users. An overview of the computing architecture, the software tools and the distributed infrastructure deployed is reported. Summaries of the experience in establishing efficient and scalable operations to prepare for CMS distributed analysis are presented, followed by the user experience in their current analysis activities.JournalofGridComputing manuscript No. (will be inserted by the editor) Abstract CMS expects to manage several Pbytes of data each year, distributing them over many computing sites around the world and enabling data access at those centers for analysis. CMS has identified the distributed sites as the primary location for physics analysis to support a wide community of users, with potentially as many as 3000 users. This represents an unprecedented scale of distributed computing resources and number of users. An overview of the computing architecture, the software tools and the distributed infrastructure deployed is reported. Summaries of the experience in establishing efficient and scalable operations to prepare for CMS distributed analysis are presented, followed by the user experience in their current analysis activities. Distributed Analysis in CMS

show abstract

“…Today it has more than 4M lines of C++ code (CMSSW release), 2M lines of python code (framework configuration, web and data management stack), as well as various applications written in Java and perl (mostly web and data services). Major CMS data-services, such as Tier0 [4], PhEDEx [5], Data Bookkeeping System [6], Run Summary [7] are based on standard 3-tier architecture and rely on ORACLE RAC. Combined, they accumulate ∼O(100GB) of meta-data every month.…”

Section: Introductionmentioning

confidence: 99%

Life in extra dimensions of database world or penetration of NoSQL in HEP community

Kuznetsov¹,

Evans²,

Metson³

2012

J. Phys.: Conf. Ser.

Self Cite

View full text Add to dashboard Cite

Abstract. The recent buzzword in IT world is NoSQL. Major players, such as Facebook, Yahoo, Google, etc. are widely adopted different "NoSQL" solutions for their needs. Horizontal scalability, flexible data model and management of big data volumes are only a few advantages of NoSQL. In CMS experiment we use several of them in production environment. Here, we present CMS projects based on NoSQL solutions, their strengths and weaknesses as well as our experience with those tools and their coexistence with standard RDBMS solutions in our applications. IntroductionNowadays the IT infrastructure of any large organization is quite complex by default. It usually consists of heterogeneous networks, it resides in different data centers and heavily utilizes database technology to hold corporate data. Recently a new trend, the so-called NoSQL [1] solutions, joins this conglomerate. More and more companies are adding them to solve raising problems with data scalability. The range of NoSQL solutions varies from key-value stores to graph databases. Even though their advantages are controversial, their penetration into IT infrastructure is obvious. The High Energy Physics community is not an exception. Today CERN attracts most of the attention. It hosts four large experiments at LHC searching for new physics and testing our knowledge of Standard Model. One of them is Compact Muon Solenoid (CMS) [2]. More than three thousand physicists are involved in most sophisticated research program at CMS that yields a few PB of data each year at their disposal. To handle this amount of data CMS experiment relies on hundreds of computing centers around the world interconnected with each other by GRID infrastructure [3]. The CMS software has been around ten years and still is constantly evolving along with experiment requirements. Today it has more than 4M lines of C++ code (CMSSW release), 2M lines of python code (framework configuration, web and data management stack), as well as various applications written in Java and perl (mostly web and data services). Major CMS data-services, such as Tier0 [4], PhEDEx [5], Data Bookkeeping System [6], Run Summary [7] are based on standard 3-tier architecture and rely on ORACLE RAC. Combined, they accumulate ∼O(100GB) of meta-data every month. Although such infrastructure served us well in first couple of years of data taking, the application growth, big data scale and changing requirements enforced CMS developers to bring NoSQL solutions into CMS software stack. Here we discuss reasons behind this choice and our experience with

show abstract

The CMS dataset bookkeeping service

Cited by 35 publications

References 5 publications

Data Quality Monitoring for the CMS Resistive Plate Chamber Detector

Data Quality Monitoring for the CMS Resistive Plate Chamber Detector

Distributed Analysis in CMS

Life in extra dimensions of database world or penetration of NoSQL in HEP community

Contact Info

Product

Resources

About