The Pilot Way to Grid Resources Using glideinWMS

Sfiligoi, I.; Bradley, D.; Holzman, B.; Mhashilkar, Parag; Padhi, S.; Würthwein, F.

doi:10.1109/csie.2009.950

Cited by 333 publications

(247 citation statements)

References 6 publications

Supporting

Mentioning

246

Contrasting

Order By: Relevance

“…Very recently HammerCloud superseded the CMS Job Robot, so CMS now uses the same tool as ATLAS and LHCb. Still, further work is needed to achieve a more realistic representation of the site behavior, in particular by replacing the job submission via the WMS service [16] with the submission via glideinWMS [17]; in fact, only a fraction of the analysis activity still relies on the WMS and all production is done via glideins.…”

Section: Facility Monitoringmentioning

confidence: 99%

Towards a global monitoring system for CMS computing operations

Bauerdick

Sciabà

2012

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

Abstract. The operation of the CMS computing system requires a complex monitoring system to cover all its aspects: central services, databases, the distributed computing infrastructure, production and analysis workflows, the global overview of the CMS computing activities and the related historical information. Several tools are available to provide this information, developed both inside and outside of the collaboration and often used in common with other experiments. Despite the fact that the current monitoring allowed CMS to successfully perform its computing operations, an evolution of the system is clearly required, to adapt to the recent changes in the data and workload management tools and models and to address some shortcomings that make its usage less than optimal. Therefore, a recent and ongoing coordinated effort was started in CMS, aiming at improving the entire monitoring system by identifying its weaknesses and the new requirements from the stakeholders, rationalise and streamline existing components and drive future software development. This contribution gives a complete overview of the CMS monitoring system and a description of all the recent activities that have been started with the goal of providing a more integrated, modern and functional global monitoring system for computing operations. IntroductionThe CMS computing system is operating since 2008, much before the start of LHC proton-proton data taking. Its role is to enable the collaboration to process, store, distribute and analyse data collected by the CMS experiment. Like for the other LHC experiments, it is built over the infrastructure and the services provided by the Worldwide LHC Computing Grid (WLCG) as an upper layer of services and tools that take into account the CMS data model and implement the CMS computing model.The operation of such complex system requires substantial effort from CMS, from WLCG and the sites; it is therefore of utmost importance to be able to know in detail the system status and its history at all times, as the availability of good monitoring information is a prerequisite for efficient operations. In November 2010 CMS critically reviewed all the available monitoring with the goal of identifying its weaknesses and finding ways to improve it.The purpose of this paper is i) to give an overview of the CMS monitoring system, the areas covered by it, the existing tools and the known deficiencies of the system, ii) to describe the process pursued by CMS to achieve a global monitoring system based on a limited number of common tools and iii) to talk about the ongoing developments along the lines defined by the global strategy.

show abstract

Section: Facility Monitoringmentioning

confidence: 99%

Towards a global monitoring system for CMS computing operations

Bauerdick

Sciabà

2012

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

show abstract

“…This promise is tempting for the users as it lifts one of their last remaining limitations. Unfortunately, this setting leads to an unprecedented demand of resources that is often latent (e.g., users maintaining demand for resources similarly to pilot jobs in grids [8]). …”

Section: Introductionmentioning

confidence: 99%

An architecture to stimulate behavioral development of academic cloud users

Kecskeméti

Ostermann

Prodan

2014

Sustainable Computing: Informatics and Systems

View full text Add to dashboard Cite

Academic cloud infrastructures are constructed and maintained so they minimally constrain their users. Since they are free and do not limit usage patterns, academics developed such behavior that jeopardizes fair and flexible resource provisioning. For efficiency, related work either explicitly limits user access to resources, or introduce automatic rationing techniques. Surprisingly, the root cause (i.e., the user behavior) is disregarded by these approaches. This article compares academic cloud user behavior to its commercial equivalent. We deduce, that academics should behave like commercial cloud users to relieve resource provisioning. To encourage commercial like behavior, we propose an architectural extension to existing academic infrastructure clouds. First, every user's energy consumption and efficiency is monitored. Then, energy efficiency based leader boards are used to ignite competition between academics and reveal their worst practices. Leader boards are not sufficient to completely change user behavior. Thus, we introduce engaging options that encourage academics to delay resource requests and prefer resources more suitable for the infrastructure's internal provisioning. Finally, we evaluate our extensions via a simulation using real life academic resource request traces. We show a potential resource utilization reduction (by the factor of at most 2.6) while maintaining the unlimited nature of academic clouds.

show abstract

Section: Introductionmentioning

confidence: 99%

Fostering energy-awareness in scientific cloud users

Ostermann

Kecskeméti

Prodan

2014

2014 IEEE 3rd International Conference on Cloud Networking (CloudNet)

View full text Add to dashboard Cite

Abstract-Academic cloud infrastructures are constructed and maintained so they minimally constrain their users. Since they are free and do not limit usage patterns, academics developed such behavior that jeopardizes fair and flexible resource provisioning. For efficiency, related work either explicitly limits user access to resources, or introduces automatic rationing techniques. Surprisingly, the root cause (i.e., the user behavior) is disregarded by these approaches. This paper compares academic cloud user behavior to its commercial equivalent. We deduce, that academics should behave like commercial cloud users to relieve resource provisioning. To encourage this behavior, we propose an architectural extension to academic infrastructure clouds. We evaluate our extension via a simulation using real life academic resource request traces. We show a potential resource usage reduction while maintaining the unlimited nature of academic clouds.

show abstract

The Pilot Way to Grid Resources Using glideinWMS

Cited by 333 publications

References 6 publications

Towards a global monitoring system for CMS computing operations

Towards a global monitoring system for CMS computing operations

An architecture to stimulate behavioral development of academic cloud users

Fostering energy-awareness in scientific cloud users

Contact Info

Product

Resources

About