Experiment Dashboard for Monitoring of the LHC Distributed Computing Systems

Andreeva, Julia; Campos, M Devesas; Cros, J Tarragon; Gaidioz, Benjamin; Karavakis, E.; Kokoszkiewicz, Lukasz; Lanciotti, E.; Maier, Gerhild; Ollivier, W; Nowotka, Michał; Rocha, Ricardo; Sadykov, T. Kh.; Sáiz, Pablo; Sargsyan, L; Sidorova, I. B.; Tuckett, David

doi:10.1088/1742-6596/331/7/072001

Cited by 7 publications

(5 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then different information about the jobs is uploaded to different CMS services. The job status is uploaded to Dashboard [10] and the job output (files) information is uploaded to PhEDeX (Data Transfers) [6] and DBS (Data Bookkeeping System) [7].…”

Section: Production Setupmentioning

confidence: 99%

A new era for central processing and production in CMS

Fajardo

Gutsche

Foulkes

et al. 2012

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

The goal for CMS computing is to maximise the throughput of simulated event generation while also processing the real data events as quickly and reliably as possible. To maintain this achievement as the quantity of events increases, since the beginning of 2011 CMS computing has migrated at the Tier 1 level from its old production framework, ProdAgent, to a new one, WMAgent. The WMAgent framework offers improved processing efficiency and increased resource usage as well as a reduction in manpower. In addition to the challenges encountered during the design of the WMAgent framework, several operational issues have arisen during its commissioning. The largest operational challenges were in the usage and monitoring of resources, mainly a result of a change in the way work is allocated. Instead of work being assigned to operators, all work is centrally injected and managed in the Request Manager system and the task of the operators has changed from running individual workflows to monitoring the global workload. In this report we present how we tackled some of the operational challenges, and how we benefitted from the lessons learned in the commissioning of the WMAgent framework at the Tier 2 level in late 2011. As case studies, we will show how the WMAgent system performed during some of the large data reprocessing and Monte Carlo simulation campaigns.

show abstract

Section: Production Setupmentioning

confidence: 99%

A new era for central processing and production in CMS

Fajardo

Gutsche

Foulkes

et al. 2012

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

show abstract

“…It is a job-centric view aimed at facilitating understanding and debugging of job processing in real-time [7]. The entry point is the number of jobs submitted or accomplished in a chosen time period [ fig.…”

Section: Interactive Viewmentioning

confidence: 99%

hBrowse - Generic framework for hierarchical data visualization

Andreeva¹,

Dzhunov²,

Karavakis³

et al. 2012

Proceedings of EGI Community Forum 2012 / EMI Second Technical Conference — PoS(EGICF12-EMITC2)

View full text Add to dashboard Cite

“…Examples are the PhEDEx monitoring [2], the CRAB server monitoring [3] and the WMAgent monitoring [4]; • a global overview of the CMS computing services is provided via SLS, the monitoring information aggregation and visualization tool developed and used by the CERN IT for the computing centre [5]. The status of computing services as measured by SLS is further aggregated in the Critical Services map developed by the Dashboard team [6].…”

Section: Service Monitoringmentioning

confidence: 99%

“…End users submitting analysis jobs are served by the CMS Dashboard applications, in particular the Interactive View and the Task Monitoring [6]. Both were developed by the CERN IT, first for CMS and then ported to ATLAS.…”

Section: Global Overviewmentioning

confidence: 99%

Towards a global monitoring system for CMS computing operations

Bauerdick

Sciabà

2012

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

Abstract. The operation of the CMS computing system requires a complex monitoring system to cover all its aspects: central services, databases, the distributed computing infrastructure, production and analysis workflows, the global overview of the CMS computing activities and the related historical information. Several tools are available to provide this information, developed both inside and outside of the collaboration and often used in common with other experiments. Despite the fact that the current monitoring allowed CMS to successfully perform its computing operations, an evolution of the system is clearly required, to adapt to the recent changes in the data and workload management tools and models and to address some shortcomings that make its usage less than optimal. Therefore, a recent and ongoing coordinated effort was started in CMS, aiming at improving the entire monitoring system by identifying its weaknesses and the new requirements from the stakeholders, rationalise and streamline existing components and drive future software development. This contribution gives a complete overview of the CMS monitoring system and a description of all the recent activities that have been started with the goal of providing a more integrated, modern and functional global monitoring system for computing operations. IntroductionThe CMS computing system is operating since 2008, much before the start of LHC proton-proton data taking. Its role is to enable the collaboration to process, store, distribute and analyse data collected by the CMS experiment. Like for the other LHC experiments, it is built over the infrastructure and the services provided by the Worldwide LHC Computing Grid (WLCG) as an upper layer of services and tools that take into account the CMS data model and implement the CMS computing model.The operation of such complex system requires substantial effort from CMS, from WLCG and the sites; it is therefore of utmost importance to be able to know in detail the system status and its history at all times, as the availability of good monitoring information is a prerequisite for efficient operations. In November 2010 CMS critically reviewed all the available monitoring with the goal of identifying its weaknesses and finding ways to improve it.The purpose of this paper is i) to give an overview of the CMS monitoring system, the areas covered by it, the existing tools and the known deficiencies of the system, ii) to describe the process pursued by CMS to achieve a global monitoring system based on a limited number of common tools and iii) to talk about the ongoing developments along the lines defined by the global strategy.

show abstract

Experiment Dashboard for Monitoring of the LHC Distributed Computing Systems

Cited by 7 publications

References 1 publication

A new era for central processing and production in CMS

A new era for central processing and production in CMS

hBrowse - Generic framework for hierarchical data visualization

Towards a global monitoring system for CMS computing operations

Contact Info

Product

Resources

About