Temporal Models on Time Series Databases.

Mazak, Alexandra; Wolny, Sabine; Gomez, A.E.; Cabot, Jordi; Wimmer, Manuel; Kappel, Gerti

doi:10.5381/jot.2020.19.3.a14

Cited by 11 publications

(4 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…TMs go beyond representing and processing the current state of systems [6]. They seek to add short and long-term memory to models through the use of temporal databases [15]. Examples of temporal databases used for TMs, are Time Series Databases (TSDB) and Temporal Graph Databases (TGDB) [15,21].…”

Section: Temporal Modelsmentioning

confidence: 99%

“…They seek to add short and long-term memory to models through the use of temporal databases [15]. Examples of temporal databases used for TMs, are Time Series Databases (TSDB) and Temporal Graph Databases (TGDB) [15,21]. Each attribute to be monitored in a running system can be considered as a time series: a sequence of values along an axis [3].…”

Section: Temporal Modelsmentioning

confidence: 99%

See 1 more Smart Citation

A Framework for History-Aware Hyperparameter Optimisation in Reinforcement Learning

Parra-Ullauri¹,

Жен²,

García-Domínguez³

et al. 2023

Preprint

View full text Add to dashboard Cite

A Reinforcement Learning (RL) system depends on a set of initial conditions (hyperparameters) that affect the system's performance. However, defining a good choice of hyperparameters is a challenging problem. Hyperparameter tuning often requires manual or automated searches to find optimal values. Nonetheless, a noticeable limitation is the high cost of algorithm evaluation for complex models, making the tuning process computationally expensive and time-consuming. In this paper, we propose a framework based on integrating complex event processing and temporal models, to alleviate these trade-offs. Through this combination, it is possible to gain insights about a running RL system efficiently and unobtrusively based on data stream monitoring and to create abstract representations that allow reasoning about the historical behaviour of the RL system. The obtained knowledge is exploited to provide feedback to the RL system for optimising its hyperparameters while making effective use of parallel resources. We introduce a novel history-aware epsilon-greedy logic for hyperparameter optimisation that instead of using static hyperparameters that are kept fixed for the whole training, adjusts the hyperparameters at runtime based on the analysis of the agent's performance over time windows in a single agent's lifetime. We tested the proposed approach in a 5G mobile communications case study that uses DQN, a variant of RL, for its decision-making. Our experiments demonstrated the effects of hyperparameter tuning using history on training stability and reward values. The encouraging results show that the proposed history-aware framework significantly improved performance compared to traditional hyperparameter tuning approaches.

show abstract

Section: Temporal Modelsmentioning

confidence: 99%

Section: Temporal Modelsmentioning

confidence: 99%

A Framework for History-Aware Hyperparameter Optimisation in Reinforcement Learning

Parra-Ullauri¹,

Жен²,

García-Domínguez³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Mazak et al [39] proposed a runtime monitoring solution in which models can be partially mapped to time series databases. The solution is able to collect runtime information (i.e.…”

Section: Runtime Monitoringmentioning

confidence: 99%

Event-driven temporal models for explanations - ETeMoX: explaining reinforcement learning

et al. 2021

View full text Add to dashboard Cite

Modern software systems are increasingly expected to show higher degrees of autonomy and self-management to cope with uncertain and diverse situations. As a consequence, autonomous systems can exhibit unexpected and surprising behaviours. This is exacerbated due to the ubiquity and complexity of Artificial Intelligence (AI)-based systems. This is the case of Reinforcement Learning (RL), where autonomous agents learn through trial-and-error how to find good solutions to a problem. Thus, the underlying decision-making criteria may become opaque to users that interact with the system and who may require explanations about the system’s reasoning. Available work for eXplainable Reinforcement Learning (XRL) offers different trade-offs: e.g. for runtime explanations, the approaches are model-specific or can only analyse results after-the-fact. Different from these approaches, this paper aims to provide an online model-agnostic approach for XRL towards trustworthy and understandable AI. We present ETeMoX, an architecture based on temporal models to keep track of the decision-making processes of RL systems. In cases where the resources are limited (e.g. storage capacity or time to response), the architecture also integrates complex event processing, an event-driven approach, for detecting matches to event patterns that need to be stored, instead of keeping the entire history. The approach is applied to a mobile communications case study that uses RL for its decision-making. In order to test the generalisability of our approach, three variants of the underlying RL algorithms are used: Q-Learning, SARSA and DQN. The encouraging results show that using the proposed configurable architecture, RL developers are able to obtain explanations about the evolution of a metric, relationships between metrics, and were able to track situations of interest happening over time windows.

show abstract

“…Hawk [44], to which InTempo was compared, uses the time-aware graph database in [59]. The solutions in [51] and [80] use a map-and time-series-database, respectively. As the evaluation in Sect.…”

Section: Related Workmentioning

confidence: 99%

Incremental execution of temporal graph queries over runtime models with history and its applications

et al. 2021

View full text Add to dashboard Cite

Modern software systems are intricate and operate in highly dynamic environments for which few assumptions can be made at design-time. This setting has sparked an interest in solutions that use a runtime model which reflects the system state and operational context to monitor and adapt the system in reaction to changes during its runtime. Few solutions focus on the evolution of the model over time, i.e., its history, although history is required for monitoring temporal behaviors and may enable more informed decision-making. One reason is that handling the history of a runtime model poses an important technical challenge, as it requires tracing a part of the model over multiple model snapshots in a timely manner. Additionally, the runtime setting calls for memory-efficient measures to store and check these snapshots. Following the common practice of representing a runtime model as a typed attributed graph, we introduce a language which supports the formulation of temporal graph queries, i.e., queries on the ordering and timing in which structural changes in the history of a runtime model occurred. We present a querying scheme for the execution of temporal graph queries over history-aware runtime models. Features such as temporal logic operators in queries, the incremental execution, the option to discard history that is no longer relevant to queries, and the in-memory storage of the model, distinguish our scheme from relevant solutions. By incorporating temporal operators, temporal graph queries can be used for runtime monitoring of temporal logic formulas. Building on this capability, we present an implementation of the scheme that is evaluated for runtime querying, monitoring, and adaptation scenarios from two application domains.

show abstract

Temporal Models on Time Series Databases.

Cited by 11 publications

References 29 publications

A Framework for History-Aware Hyperparameter Optimisation in Reinforcement Learning

A Framework for History-Aware Hyperparameter Optimisation in Reinforcement Learning

Event-driven temporal models for explanations - ETeMoX: explaining reinforcement learning

Incremental execution of temporal graph queries over runtime models with history and its applications

Contact Info

Product

Resources

About