Scheduling In-Situ Analytics in Next-Generation Applications

Mondragon, Oscar H.; Bridges, Patrick G.; Levy, Scott; Ferreira, Kurt Brian; Widener, Patrick

doi:10.1109/ccgrid.2016.42

Cited by 3 publications

(3 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the work of Mondragon et al, the authors explored the problem of deciding the nodes on which data analytics may be run. Their intuition is that executing analytics as well as simulation on the same computation node leads to a reduction in time and energy consumption.…”

Section: Related Workmentioning

confidence: 99%

Load balancing aware scheduling algorithms for fog networks

Singh

Auluck

2019

Softw Pract Exp

View full text Add to dashboard Cite

Fog networks have attracted the attention of researchers recently. The idea is that a part of the computation of a job/application can be performed by fog devices that are located at the network edge, close to the users. Executing latency sensitive applications on the cloud may not be feasible, owing to the significant communication delay involved between the user and the cloud data center (cdc). By the time the application traverses the network and reaches the cloud data center, it might already be too late. However, fog devices, also known as mobile data centers (mdcs), are capable of executing such latency sensitive applications. In this paper, we study the problem of balancing the application load while taking account of security constraints of jobs, across various mdcs in a fog network. In case a particular mdc does not have sufficient capacity to execute a job, the job needs to be migrated to some other mdc. To this end, we propose three heuristic algorithms: minimum distance, minimum load, and minimum hop distance and load (MHDL). In addition, we also propose an ILP-based algorithm called load balancing aware scheduling ILP (LASILP) for solving the task mapping and scheduling problem. The performance of the proposed algorithms have been compared with the cloud only algorithm and another heuristic algorithm called fog-cloud-placement (FCP). Simulation results performed on real-life workload traces reveal that the MHDL heuristic performs better as compared to other scheduling policies in the fog computing environment while meeting application privacy requirements.

show abstract

Section: Related Workmentioning

confidence: 99%

Load balancing aware scheduling algorithms for fog networks

Singh

Auluck

2019

Softw Pract Exp

View full text Add to dashboard Cite

show abstract

“…However, as node counts increase, the likelihood of one task experiencing an interruption becomes a key factor to performance; as the BSP per‐loop iteration time decreases, the effects of even smaller noise perturbations increase, further emphasizing the usefulness of tightly synchronized clocks. () To mitigate the potentially disastrous hit on performance, sophisticated runtime systems and operating systems seek to overlap any disruption in progress across all compute nodes. When system software is able to schedule all interrupting tasks on every node such that they occur simultaneously, the cascading effect on the parallel algorithm is kept in check—a strategy known as coordinated scheduling .…”

Section: Importance Of Time Agreementmentioning

confidence: 99%

“…The scheduler must guarantee precise CPU reservations for cooperative codes and accurate timing to run applications with coordination needs across nodes. () In this scenario, a precise inter‐node time agreement is critical. Moreover, the absence of suitable time agreement prevents effective gang‐scheduling and coordinated‐scheduling.…”

Section: Importance Of Time Agreementmentioning

confidence: 99%

An evaluation of the state of time synchronization on leadership class supercomputers

Jones

Ostrouchov

Koenig

et al. 2017

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

SummaryWe present a detailed examination of time agreement characteristics for nodes within extreme-scale parallel computers. Using a software tool we introduce in this paper, we quantify attributes of clock skew among nodes in three representative high-performance computers sited at three national laboratories. Our measurements detail the statistical properties of time agreement among nodes and how time agreement drifts over typical application execution durations.We discuss the implications of our measurements, why the current state of the field is inadequate, and propose strategies to address observed shortcomings. KEYWORDSclock synchronization, large-scale systems, system software, time service INTRODUCTIONThe trend towards increasing node counts in high-performance computing (HPC) is motivating a move toward greater levels of concurrency in HPC systems. Today's software environment is now being called on to produce new solutions for emerging issues including managing system power, resilience, and performance characteristics. The distributed algorithms that underlie such services operate much more efficiently in the presence of tightly synchronized clocks. For example, tightly synchronized clocks benefit well-known gang scheduling techniques and complex consensus algorithms. To illustrate the point, such time synchronization enables more aggressive assumptions about communication and synchronization patterns, the removal of unnecessary locks, and a wide range of other applications. Clock-based techniques are already frequently deployed in cloud and data center distributed systems for precisely these reasons.We examined the time synchronization on some of the world's fastest and most powerful machines. These leadership-class systems employ high-end hardware connected by an extremely low-latency, low-jitter, interconnect in a carefully controlled environment, in contrast to widely distributed cloud-based systems based on commodity hardware and networks. Because of this, we assumed that these systems would have more stable, predictable hardware clocks, and close base time agreement using only standard time synchronization systems like Network Time Protocol (NTP). We did not believe that the complex hardware and software techniques used to provide time synchronization in wide-area systems would be necessary in leadership systems.Our results demonstrate that the actual time uncertainty for leadership-class machines is often unexpectedly large, in some cases over 600 milliseconds despite network latencies of less than two microseconds. Building on this, we set out to thoroughly quantify the magnitude of the time synchronization challenge in leadership-class systems. This study shows that the current time protocol in use, NTP, is not suitable for providing the level of time synchronization necessary for important system software tasks such as coordinated scheduling. Based on this, we conclude

show abstract