DILoS: A dynamic integrated load manager and scheduler for continuous queries

Pham, Thao Nguyen; Moakar, Lory Al; Chrysanthis, Panos K.; Labrinidis, Alexandros

doi:10.1109/icdew.2011.5767652

Cited by 8 publications

(8 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As opposed to the above works, DILoS, which was initially proposed in a 6-page workshop paper [37] and expanded and enhanced in this paper, consistently combines priority-based scheduling and load shedding to provide a guarantee on QoS and class-consistent QoD.…”

Section: Related Workmentioning

confidence: 93%

See 1 more Smart Citation

Avoiding class warfare: managing continuous queries with differentiated classes of service

2015

Self Cite

View full text Add to dashboard Cite

Data stream management systems (DSMSs) offer the most effective solution for processing data streams by efficiently executing continuous queries (CQs) over the incoming data. CQs inherently have different levels of criticality and hence different levels of expected quality of service (QoS) and quality of data (QoD). Adhering to such expected QoS/QoD metrics is even more important in cases of multi-tenant data stream management services. In this work, we propose DILoS, a framework that, through prioritybased scheduling and load shedding, supports differentiated QoS and QoD for multiple classes of CQs. Unlike existing works that consider scheduling and load shedding separately, DILoS is a novel unified framework that exploits the synergy between scheduling and load shedding. We also propose ALoMa, a general, adaptive load manager that DILoS is built upon. By its design, ALoMa performs better than the state-ofthe-art alternatives in three dimensions: (1) it automatically tunes the headroom factor, (2) it honors the delay target, (3) it is applicable to complex query networks with shared operators. We implemented DILoS and ALoMa in our real DSMS prototype system (AQSIOS) and evaluate their performance for a variety of real and synthetic workloads. Our experimental evaluation of ALoMa verified its clear superiority over the This article enhances and extends preliminary work [37] that was presented in the SMDB'11 Workshop.state-of-the-art approaches. Our experimental evaluation of the DILoS framework showed that it (a) allows the scheduler and load shedder to consistently honor CQs' priorities, (b) significantly increases system capacity utilization by exploiting batch processing, and (c) enables operator sharing among query classes of different priorities while avoiding priority inversion, i.e., a lower-priority class never blocks a higherpriority one.

show abstract

Section: Related Workmentioning

confidence: 93%

“…We proposed the idea of DILoS in a short paper in the SMDB'11 workshop [37]. To the best of our knowledge, we were the first to identify and analyze the problem of integrating a priority-aware scheduler and load manager in a DSMS.…”

Section: Introductionmentioning

confidence: 99%

Avoiding class warfare: managing continuous queries with differentiated classes of service

2015

Self Cite

View full text Add to dashboard Cite

show abstract

“…CQC aims to minimize the latency of high-priority queries and maintain reasonable latency values for the low-priority ones. Pham et al extend this work and explore the relationship between scheduling and load management in [19,20]. Their scheduler and load manager work in synergy, exchanging runtime information to consistently honor the user-defined priorities of the queries while increasing the system's utilization.…”

Section: Related Workmentioning

confidence: 99%

Haren

Palyvos-Giannas

Gulisano

Papatriantafilou

2019

Proceedings of the 13th ACM International Conference on Distributed and Event-Based Systems

View full text Add to dashboard Cite

In modern Stream Processing Engines (SPEs), numerous diverse applications, which can differ in aspects such as cost, criticality or latency sensitivity, can co-exist in the same computing node. When these differences need to be considered to control the performance of each application, custom scheduling of operators to threads is of key importance (e.g., when a smart vehicle needs to ensure that safety-critical applications always have access to computational power, while other applications are given lower, variable priorities). Many solutions have been proposed regarding schedulers that allocate threads to operators to optimize specific metrics (e.g., latency) but there is still lack of a tool that allows arbitrarily complex scheduling strategies to be seamlessly plugged on top of an SPE. We propose Haren to fill this gap. More specifically, we (1) formalize the thread scheduling problem in stream processing in a general way, allowing to define ad-hoc scheduling policies, (2) identify the bottlenecks and the opportunities of scheduling in stream processing, (3) distill a compact interface to connect Haren with SPEs, enabling rapid testing of various scheduling policies, (4) illustrate the usability of the framework by integrating it into an actual SPE and (5) provide a thorough evaluation. As we show, Haren makes it is possible to adapt the use of computational resources over time to meet the goals of a variety of scheduling policies. CCS CONCEPTS • Information systems → Online analytical processing engines; • Software and its engineering → Scheduling; Middleware.

show abstract

“…Many multimedia services are provided through the same CDN system simultaneously, and it is expected that each multimedia application and content request (from users) made to the corresponding multimedia application will demand different QoS requirements when making use of the service. To encompass the needs of service users, while producing sufficient profit, a CDN provider can offer different grades of service, which are typically described in a service-level agreement (SLA), and the service cost is determined by the SLA chosen by the user [13], [14], [16].…”

Section: Dynamic Resource Allocation and Scheduling For Cloud-based Vmentioning

confidence: 99%

Dynamic Resource Allocation and Scheduling for Cloud-Based Virtual Content Delivery Networks

Lee

Ryu

et al. 2014

ETRI J

View full text Add to dashboard Cite

DILoS: A dynamic integrated load manager and scheduler for continuous queries

Cited by 8 publications

References 12 publications

Avoiding class warfare: managing continuous queries with differentiated classes of service

Avoiding class warfare: managing continuous queries with differentiated classes of service

Haren

Dynamic Resource Allocation and Scheduling for Cloud-Based Virtual Content Delivery Networks

Contact Info

Product

Resources

About