No abstract

No abstract

Abstract-In this paper we investigate sequential decision mechanisms for composite web services. After executing each sub-service within a sequential workflow, decisions are made whether to terminate or continue the execution of the workflow. These decisions are based on observed response times, expected rewards, and typical Service Level Agreement parameters such as costs, penalties, and agreed response-time objectives. We propose a model for the sequential decision-making process within which we explore a couple of decision algorithms. We benchmarked these algorithms against the profit made when executing the workflow without decision-making. We show that algorithm based on backward recursion principle of dynamic programming is optimal with respect to profit. Next, we analyse the structure of erroneous decisions for both algorithms and show that significant profit gains can be obtained by sequential decision making.

We consider a general class of dynamic resource allocation problems within a stochastic optimal control framework. This class of problems arises in a wide variety of applications, each of which intrinsically involves resources of different types and demand with uncertainty and/or variability. The goal is to dynamically allocate capacity for every resource type in order to serve the uncertain/variable demand and maximize the expected net-benefit over a time horizon of interest based on the rewards and costs associated with the different resources. We derive the optimal control policy within a singular control setting, which includes easily implementable algorithms for governing the dynamic adjustments to resource allocation capacities over time. Numerical experiments investigate various issues of both theoretical and practical interest, quantifying the significant benefits of our approach over alternative optimization approaches.

Large organizations like banks suffer from the ever growing complexity of their systems. Evolving the software becomes harder and harder since a single change can affect a much larger part of the system than predicted upfront. A large contributing factor to this problem is that the actual domain knowledge is often implicit, incomplete, or out of date, making it difficult to reason about the correct behavior of the system as a whole. With Rebel we aim to capture and centralize the domain knowledge and relate it to the running systems.Rebel is a formal specification language for controlling the intrinsic complexity of software for financial enterprise systems. In collaboration with ING, a large Dutch bank, we developed the Rebel specification language and an Integrated Specification Environment (ISE), currently offering automated simulation and checking of Rebel specifications using a Satisfiability Modulo Theories (SMT) solver.In this paper we report on our design choices for Rebel, the implementation and features of the ISE, and our initial observations on the application of Rebel inside the bank.

The advent of renewable energy sources has huge implications for the design and control of power grids. On the engineering side, reliability is currently ensured by strict con- straints on current, voltage and temperature. However, with growing supply-side uncertainty induced by renewables, these will need to be replaced by probabilistic guarantees, allowing constraints on a given line to be violated with a low probability, e.g., several minutes per year. In the present note we illustrate, using large deviations techniques, how replacing (probabilistic) current constraints by temperature constraints can lead to capacity gains in power grids.

Language systems have been of great interest to the research community and have recently reached the mass market through various assistant platforms on the web. Reinforcement Learning methods that optimize dialogue policies have seen successes in past years and have recently been extended into methods that personalize the dialogue, e.g. take the personal context of users into account. These works, however, are limited to personalization to a single user with whom they require multiple interactions and do not generalize the usage of context across users. This work introduces a problem where a generalized usage of context is relevant and proposes two Reinforcement Learning (RL)-based approaches to this problem. The first approach uses a single learner and extends the traditional POMDP formulation of dialogue state with features that describe the user context. The second approach segments users by context and then employs a learner per context. We compare these approaches in a benchmark of existing non-RL and RL-based methods in three established and one novel application domain of financial product recommendation. We compare the influence of context and training experiences on performance and find that learning approaches generally outperform a handcrafted gold standard.

Abstract-Service orchestration has become the predominant paradigm that enables businesses to combine and integrate services offered by third parties. For the commercial viability of orchestrated services, it is crucial that they are offered at sharp price-quality ratios. A complicating factor is that many attractive third-party services often show highly variable service quality. This raises the need for mechanisms that promptly adapt the orchestration to changes in the quality delivered by third party services.In this paper, we propose a real-time QoS control mechanism that dynamically optimizes service orchestration in real time by learning and adapting to changes in third party service response time behaviors. Our approach combines the power of learning and adaptation with the power of dynamic programming. The results show that real-time service re-compositions lead to dramatic savings of cost, while meeting the service quality requirements of the end-users. The challenge here is to respond to significant response-time changes in a timely manner, while not wasting CPU cycles on unnecessary orchestration updates. Experimental results performed in a test-lab environment demonstrate that a few orchestration updates are sufficient to achieve this.