The dark side of event sourcing: Managing data conversion

Overeem, Michiel; Spoor, Marten; Jansen, Slinger

doi:10.1109/saner.2017.7884621

Cited by 17 publications

(23 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In regard to the event-driven and CEP-based approaches for runtime monitoring, Fowler [23] proposed an event sourcing model that facilitates the traceability of the changes over time of the application state as an event sequence. However, event sourcing can be costly in terms of performance [45] since this model tracks every change leading up to a state. The present work describes an event-driven approach integrating CEP to both monitor event streams efficiently, and also deal with scalability problems through the use of temporal graphs for historical data.…”

Section: Runtime Monitoringmentioning

confidence: 99%

Event-driven temporal models for explanations - ETeMoX: explaining reinforcement learning

et al. 2021

View full text Add to dashboard Cite

Modern software systems are increasingly expected to show higher degrees of autonomy and self-management to cope with uncertain and diverse situations. As a consequence, autonomous systems can exhibit unexpected and surprising behaviours. This is exacerbated due to the ubiquity and complexity of Artificial Intelligence (AI)-based systems. This is the case of Reinforcement Learning (RL), where autonomous agents learn through trial-and-error how to find good solutions to a problem. Thus, the underlying decision-making criteria may become opaque to users that interact with the system and who may require explanations about the system’s reasoning. Available work for eXplainable Reinforcement Learning (XRL) offers different trade-offs: e.g. for runtime explanations, the approaches are model-specific or can only analyse results after-the-fact. Different from these approaches, this paper aims to provide an online model-agnostic approach for XRL towards trustworthy and understandable AI. We present ETeMoX, an architecture based on temporal models to keep track of the decision-making processes of RL systems. In cases where the resources are limited (e.g. storage capacity or time to response), the architecture also integrates complex event processing, an event-driven approach, for detecting matches to event patterns that need to be stored, instead of keeping the entire history. The approach is applied to a mobile communications case study that uses RL for its decision-making. In order to test the generalisability of our approach, three variants of the underlying RL algorithms are used: Q-Learning, SARSA and DQN. The encouraging results show that using the proposed configurable architecture, RL developers are able to obtain explanations about the evolution of a metric, relationships between metrics, and were able to track situations of interest happening over time windows.

show abstract

Section: Runtime Monitoringmentioning

confidence: 99%

Event-driven temporal models for explanations - ETeMoX: explaining reinforcement learning

et al. 2021

View full text Add to dashboard Cite

show abstract

“…This data storage model keeps track of every data change without removing earlier events [33]. The stored event streams can be queried on demand for monitoring [27] or retroactive computing purposes [26]. A problem with event sourcing is that keeping track of not only the current state, but also every change leading up to that state could be costly on terms of performance [27].…”

Section: Related Work 51 Event-driven Approaches For Runtime Monitoringmentioning

confidence: 99%

“…The stored event streams can be queried on demand for monitoring [27] or retroactive computing purposes [26]. A problem with event sourcing is that keeping track of not only the current state, but also every change leading up to that state could be costly on terms of performance [27]. On the contrary, the present paper describes an event-driven approach with the use of CEP for efficient event streams monitoring and deal with scalability problems through the use of temporal graphs for historical data.…”

Section: Related Work 51 Event-driven Approaches For Runtime Monitoringmentioning

confidence: 99%

Towards an architecture integrating complex event processing and temporal graphs for service monitoring

Parra-Ullauri

García-Domínguez

Boubeta-Puig

et al. 2021

Proceedings of the 36th Annual ACM Symposium on Applied Computing

View full text Add to dashboard Cite

Software is becoming more complex as it needs to deal with an increasing number of aspects in volatile environments. This complexity may cause behaviors that violate the imposed constraints. A goal of runtime service monitoring is to determine whether the service behaves as intended to potentially allow the correction of the behavior. It may be set up in advance the infrastructure to allow the detections of suspicious situations. However, there may also be unexpected situations to look for as they only become evident during data stream monitoring at runtime produced by te system. The access to historic data may be key to detect relevant situations in the monitoring infrastructure. Available technologies used for monitoring offer different trade-offs, e.g. in cost and flexibility to store historic information. For instance, Temporal Graphs (TGs) can store the long-term history of an evolving system for future querying, at the expense of disk space and processing time. In contrast, Complex Event Processing (CEP) can quickly react to incoming situations efficiently, as long as the appropriate event patterns have been set up in advance. This paper presents an architecture that integrates CEP and TGs for service monitoring through the data stream produced at runtime by a system. The pros and cons of the proposed architecture for extracting and treating the monitored data are analyzed. The approach is applied on the monitoring of Quality of Service (QoS) of a data-management network case study. It is demonstrated how the architecture provides rapid detection of issues, as well as the ability to access to historical data about the state of the system to allow for a comprehensive monitoring solution. CCS CONCEPTS• Software and its engineering → Maintaining software.

show abstract

“…Using the snapshot architecture, the current state of an order can be stored in a row of a relational table Order. When using the event sourcing (log) [17], the application persists each order as a sequence of events e.g., listing the creation of the order, its update with customer details and the addition of each line item.…”

Section: Microservices As a Federated Multidatabasementioning

confidence: 99%

Microservice Disaster Crash Recovery: A Weak Global Referential Integrity Management

Manouvrier

Pautasso

Rukoz

2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Microservices which use polyglot persistence (using multiple data storage techniques) cannot be recovered in a consistent state from backups taken independently. As a consequence, references across microservice boundaries may break after disaster recovery. In this paper, we give a weak global consistency definition for microservice architectures and present a recovery protocol which takes advantage of cached referenced data to reduce the amnesia interval for the recovered microservice, i.e., the time interval after the most recent backup, during which state changes may have been lost.

show abstract

The dark side of event sourcing: Managing data conversion

Cited by 17 publications

References 22 publications

Event-driven temporal models for explanations - ETeMoX: explaining reinforcement learning

Event-driven temporal models for explanations - ETeMoX: explaining reinforcement learning

Towards an architecture integrating complex event processing and temporal graphs for service monitoring

Microservice Disaster Crash Recovery: A Weak Global Referential Integrity Management

Contact Info

Product

Resources

About