A MapReduce task scheduling algorithm for deadline constraints

Tang, Zhuo; Zhou, Junqing; Li, Kenli; Li, Ruixuan

doi:10.1007/s10586-012-0236-5

Cited by 59 publications

(18 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, some authors [15], [16] have worked on the definition of a real-time version of map-reduce from a practical perspective. For instance, [15] has analyzed the problem using a constraint satisfaction problem and has introduced several heuristic strategies for this formulation.…”

Section: Real-time Parallel Computingmentioning

confidence: 99%

Patterns for Distributed Real-Time Stream Processing

Val

Fernandez-Garcia

Sánchez-Fernandez

et al. 2017

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

Abstract-In recent yeais, big data systems have become an active area of research and development. Stream processing is one of the potential application scenarios of big data systems where the goal is to process aconlinoous, high velochyflow of information items. High frequencytradirg (HFT) in stock markets ortrendirgtopicdetection in Twitter are som e examples of stream processing applications. In some cases (like, for instance, in HFT), these applications have end-t�nd qualhy-of-service reqlirements and may benefh from the usage� real-time techniques. Taking this into account, the present articl e analyzes, from the point� view of real-time systems, a set of patterns that can be used when implementing a stream processing application. For each pattern, we discuss its advantages and dsadvanlages, as well as its impact in application performance, measured as response time, maximum ill)Ut frequency and changes in utilization demands due to the pattern.

show abstract

Section: Real-time Parallel Computingmentioning

confidence: 99%

Patterns for Distributed Real-Time Stream Processing

Val

Fernandez-Garcia

Sánchez-Fernandez

et al. 2017

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

show abstract

“…For the default method of Hadoop, which cannot schedule the tasks to the nodes with the prefetched data, a prefetching technique was proposed in [17] to hide the remote data access delay caused by the map tasks processed on the nodes without the input data. We proposed MTSD, an extensional MapReduce task scheduling algorithm for deadline constraints in the Hadoop platform [18], which allows a user to specify a job's deadline and tries to make the job to be finished before the deadline. Some of these proposals [19] presented the workload characteristic oriented scheduler, which strives for co-locating tasks of possibly different MapReduce jobs with complementing resource usage characteristics.…”

Section: Related Workmentioning

confidence: 99%