MonALISA: An agent based, dynamic service system to monitor, control and optimize distributed systems

Legrand, I.; Newman, H. B.; Voicu, Ramiro; Cîrstoiu, Cătălin; Grigoras, C.; Dobre, Ciprian; Muraru, Adrian; Costan, Alexandru; Dediu, M.; Stratan, Corina

doi:10.1016/j.cpc.2009.08.003

Cited by 113 publications

(81 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Being able to become aware of the overall network state on a given link in a given moment, and avoid to make transfer choices on the basis of PhEDEx-internal statistics instead, as well as refraining from running real-time heartbeat-like tests, it would offer large dividends: PhEDEx could "passively" access and use network information (e.g. PerfSONAR [31], MonaLISA [32]) and use them to tune its transfer activity. On the other hand, CMS could "actively" use the network information, i.e.…”

Section: Network Information and Analysis Throughputmentioning

confidence: 99%

Challenging data management in CMS computing with network-aware systems

Bonacorsi

Wildish

2013

2013 IEEE Nuclear Science Symposium and Medical Imaging Conference (2013 NSS/MIC)

View full text Add to dashboard Cite

After a successful first run at the LHC, and during the Long Shutdown (LS1) of the accelerator, the workload and data management sectors of the CMS Computing Model are entering into an operational review phase in order to concretely assess area of possible improvements and paths to exploit new promising technology trends. In particular, since the preparation activities for the LHC start, the Networks have constantly been of paramount importance for the execution of CMS workflows, exceeding the original expectations -as from the MONARC model -in terms of performance, stability and reliability. The low-latency transfers of PetaBytes of CMS data among dozens of WLCG Tiers worldwide using the PhEDEx dataset replication system is an example of the importance of reliable Networks. Another example is the exploitation of WAN data access over data federations in CMS. A new emerging area of work is the exploitation of ???Intelligent Network Services??, including also bandwidth on demand concepts. In this paper, we will review the work done in CMS on this, and the next steps. Abstract-After a successful first run at the LHC, and during the Long Shutdown (LS1) of the accelerator, the workload and data management sectors of the CMS Computing Model are entering into an operational review phase in order to concretely assess area of possible improvements and paths to exploit new promising technology trends. In particular, since the preparation activities for the LHC start, the networks have constantly been of paramount importance for the execution of CMS workflows, exceeding the original expectations -as from the MONARC model -in terms of performance, stability and reliability. The low-latency transfers of PetaBytes of CMS data among dozens of WLCG Tiers worldwide using the PhEDEx dataset replication system is an example of the importance of reliable networks. Another example is the exploitation of WAN data access over data federations in CMS. A new emerging area of work is the exploitation of Intelligent Network Services, including also bandwidth on demand concepts. In this paper, we will review the work done in CMS on this, and the next steps.

show abstract

Section: Network Information and Analysis Throughputmentioning

confidence: 99%

Challenging data management in CMS computing with network-aware systems

Bonacorsi

Wildish

2013

2013 IEEE Nuclear Science Symposium and Medical Imaging Conference (2013 NSS/MIC)

View full text Add to dashboard Cite

show abstract

“…MonALISA [11] is a framework for distributed monitoring. It consists of distributed agents that handle metric monitoring for each configured host at its site and all the wide area links to other MonALISA sites.…”

Section: Related Workmentioning

confidence: 99%

Enabling persistent queries for cross-aggregate performance monitoring

et al. 2014

View full text Add to dashboard Cite

It is essential for distributed data-intensive applications to monitor the performance of the underlying network, storage and computational resources. Increasingly, distributed applications need performance information from multiple aggregates, and tools need to take real-time steering decisions based on the performance feedback. With increasing scale and complexity, the volume and velocity of monitoring data is increasing, posing scalability challenges. In this work, we have developed a Persistent Query Agent (PQA) that provides realtime application and network performance feedback to clients/applications, thereby enabling dynamic adaptations. PQA enables federated performance monitoring by interacting with multiple aggregates and performance monitoring sources. Using a publish-subscribe framework, it sends triggers asynchronously to applications/clients when relevant performance events occur. The applications/clients register their events of interest using declarative queries and get notified by the PQA. PQA leverages a complex event processing (CEP) framework for managing and executing the queries expressed in a standard SQL-like query language. Instead of saving all monitoring data for future analysis, PQA observes performance event streams in real-time, and runs continuous queries over streams of monitoring events. In this work, we present the design and architecture of the persistent query agent, and describe some relevant use cases.

show abstract

“…For instance, cientific applications such as the MonALISA [4] monitoring system for the CERN LHC Alice experiment [5]. This application collects and aggregates monitoring data from 300+ sites distributed across the world, that must be delivered to Email addresses: pmatri@fi.upm.es (Pierre Matri), mperez@fi.upm.es (María S. Pérez), alexandru.costan@irisa.fr (Alexandru Costan), luc.bouge@ens.ens-rennes.fr (Luc Bougé), gabriel.antoniu@inria.fr (Gabriel Antoniu) scientists worldwide in real-time.…”

Section: Introductionmentioning

confidence: 99%

Keeping up with storage: Decentralized, write-enabled dynamic geo-replication

Matri

Pérez

Costan

et al. 2018

Future Generation Computer Systems

View full text Add to dashboard Cite

Large-scale applications are ever-increasingly geo-distributed. Maintaining the highest possible data locality is crucial to ensure high performance of such applications. Dynamic replication addresses this problem by dynamically creating replicas of frequently accessed data close to the clients. This data is often stored in decentralized storage systems such as Dynamo or Voldemort, which offer support for mutable data. However, existing approaches to dynamic replication for such mutable data remain centralized, thus incompatible with these systems. In this paper we introduce a writeenabled dynamic replication scheme that leverages the decentralized architecture of such storage systems. We propose an algorithm enabling clients to locate tentatively the closest data replica without prior request to any metadata node. Large-scale experiments on various workloads show a read latency decrease of up to 42% compared to other state-ofthe-art, caching-based solutions.

show abstract

MonALISA: An agent based, dynamic service system to monitor, control and optimize distributed systems

Cited by 113 publications

References 1 publication

Challenging data management in CMS computing with network-aware systems

Challenging data management in CMS computing with network-aware systems

Enabling persistent queries for cross-aggregate performance monitoring

Keeping up with storage: Decentralized, write-enabled dynamic geo-replication

Contact Info

Product

Resources

About