Cost-based Fault-tolerance for Parallel Data Processing

Salama, Abdallah; Binnig, Carsten; Kraska, Tim; Zamanian, Erfan

doi:10.1145/2723372.2749437

Cited by 20 publications

(15 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• materi-all the Greenplum system that materializes all candidate operators of SIFT. • simulated-XDB an simulated-XDB system [6] on Greenplum. It regards all operators as checkpointing candidates.…”

Section: Methodsmentioning

confidence: 99%

“…Typical MPP databases model a query plan as a directed acyclic graph [6,10]. Each vertex in this figure corresponds to an operator, which is regarded as the unit of query processing.…”

Section: Parallel Query Processingmentioning

confidence: 99%

“…Moreover, execution time of today's OLAP queries has expanded a lot, due to the rapid growth of data volume. The longer a query runs, the more it suffers from system failures [6].…”

Section: Introductionmentioning

confidence: 99%

“…It should also minimize the intervention to query processing pipelines, on which MPP databases rely to achieve good performance. Some existing approaches to intra-query fault tolerance [6,10] do perform materialization selectively, aiming to maximize its cost-effectiveness. However, they either choose to block query processing pipelines [6] or hold unrealistic assumptions [10] that data transmission among query processing operators be order preserving.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Smart Intra-query Fault Tolerance for Massive Parallel Processing Databases

Chai

Zhou

et al. 2019

Data Sci. Eng.

View full text Add to dashboard Cite

Intra-query fault tolerance has increasingly been a concern for online analytical processing, as more and more enterprises migrate data analytical systems from mainframes to commodity computers. Most massive parallel processing (MPP) databases do not support intra-query fault tolerance. They may suffer from prolonged query latency when running on unreliable commodity clusters. While SQL-on-Hadoop systems can utilize the fault tolerance support of low-level frameworks, such as MapReduce and Spark, their cost-effectiveness is not always acceptable. In this paper, we propose a smart intra-query fault tolerance (SIFT) mechanism for MPP databases. SIFT achieves fault tolerance by performing checkpointing, i.e., materializing intermediate results of selected operators. Different from existing approaches, SIFT aims at promoting query success rate within a given time. To achieve its goal, it needs to: (1) minimize query rerunning time after encountering failures and (2) introduce as less checkpointing overhead as possible. To evaluate SIFT in real-world MPP database systems, we implemented it in Greenplum. The experimental results indicate that it can improve success rate of query processing effectively, especially when working with unreliable hardware.

show abstract

Section: Methodsmentioning

confidence: 99%

“…Typical MPP databases model a query plan as a directed acyclic graph [6,10]. Each vertex in this figure corresponds to an operator, which is regarded as the unit of query processing.…”

Section: Parallel Query Processingmentioning

confidence: 99%

“…Moreover, execution time of today's OLAP queries has expanded a lot, due to the rapid growth of data volume. The longer a query runs, the more it suffers from system failures [6].…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Smart Intra-query Fault Tolerance for Massive Parallel Processing Databases

Chai

Zhou

et al. 2019

Data Sci. Eng.

View full text Add to dashboard Cite

show abstract

“…In parallel data flow system FTOps [24], it showcases an optimizer that finds the best fault-tolerance strategy for each operator in a query plan. Also, a cost-based recovery [25] is proposed to select which intermediates should be materialized. These works are supplements to ours, but they focus on general workloads rather than iterative graph processing.…”

Section: A Checkpointingmentioning

confidence: 99%

Efficient fault-tolerance for iterative graph processing on distributed dataflow systems

Holzemer

Kaul

et al. 2016

2016 IEEE 32nd International Conference on Data Engineering (ICDE)

View full text Add to dashboard Cite

Real-world graph processing applications often require combining the graph data with tabular data. Moreover, graph processing usually is part of a larger analytics workflow consiting of data preparation, analysis and model building, and model application. General-purpose distributed dataflow frameworks execute all steps of such workflows holistically. This holistic view enables these systems to reason about and automatically optimize the processing. Most big graph processing algorithms are iterative and incur a long runtime, as they require multiple passes over the data until convergence. Thus, fault tolerance and quick recovery from any intermittent failure at any step of the workflow are crucial for effective and efficient analysis. In this work, we propose a novel fault-tolerance mechanism for iterative graph processing on distributed data-flow systems with the objective to reduce the checkpointing cost and failure recovery time. Rather than writing checkpoints that block downstream operators, our mechanism writes checkpoints in an unblocking manner, without breaking pipelined tasks. In contrast to the typical unblocking checkpointing approaches (i.e., managing checkpoints independently for immutable datasets), we inject the checkpoints of mutable datasets into the iterative dataflow itself. Hence, our mechanism is iteration-aware by design. This simplifies the system architecture and facilitates coordinating the checkpoint creation during iterative graph processing. We achieve speedier recovery, i.e., confined recovery, by using the local log files on each node to avoid a complete re-computation from scratch. Our theoretical studies as well as our experimental analysis on Flink give further insight into our fault-tolerance strategies and show that they are more efficient than blocking checkpointing and complete recovery for iterative graph processing on dataflow systems.

show abstract

An optimal checkpointing model with online OCI adjustment for stream processing applications

Zhuang

Wei

Li³

et al. 2019

Concurrency and Computation

View full text Add to dashboard Cite

Checkpoint-based fault-tolerant (FT) methods have been widely used to enhance the reliability of stream processing systems, but a checkpointing process usually introduces considerable overhead. It is a critical issue to choose the optimal checkpoint interval (OCI) that maximizes the processing efficiency. Traditional OCI models consider the recovery time equals to the execution time from the last checkpoint to the failure moment. However, for stream processing jobs, the recovery time is related to reprocessing workloads, depending on the real-time input data before a failure. A new model is needed to choose the OCI for stream processing applications.Moreover, the input data rate of a stream processing job fluctuates over time. To solve these problems, we present a novel DSPS OCI (DOCI) model in this paper. We prove that it maximizes the processing efficiency for a given time. We propose an approach to dynamically adjust the OCI for an application to accommodate the workload fluctuations. We conduct simulation experiments to verify the effectiveness of our DOCI model and the efficiency of the online OCI adjustment algorithm. Experimental results with a real-world dataset show that DOCI achieves an improvement on system efficiency by up to 32%, compared with existing FT approaches. KEYWORDSdistributed stream processing, fault tolerance, optimal checkpoint interval INTRODUCTIONAs users of ''big data'' applications expect fresh results, the ability to process large volumes of fast data streams in a timely fashion has become increasingly important for distributed stream processing engines (DSPEs). 1 As the DSPE frameworks are scaling to large-scale clusters, eg, Apache S4, 2 Storm, 3 Spark Streaming, 4 and Flink, 5 the stream processing applications are more vulnerable to system failures, and the fault-tolerant (FT) issue is attracting more attentions.Existing stream processing systems have managed their reliability either by active replication 6 or passive replication. 5,7 Active replication offers seamless failure recovery by switching among active backup tasks, but it introduces at least doubling resource consumptions, ie, a 50% reduction in efficiency. In contrast, passive replication can improve the system efficiency significantly, 8 which has been widely used in DSPEs for recent years. It reduces resource overhead by periodically checkpointing task processing states and saves communication overhead by combining with upstream backup. 9For DSPEs with passive replication, the optimal checkpoint interval (OCI) is the key to ensure high efficiency of stream processing applications.There is a trade-off between FT overhead and the runtime cost of failure recovery: a deficient checkpointing risk at the expense of longer failure recovery, while the frequent checkpointing incurs prohibitive impact on the normal stream processing.Concurrency Computat Pract Exper. 2019;31:e5347. wileyonlinelibrary.com/journal/cpe

show abstract

Cost-based Fault-tolerance for Parallel Data Processing

Cited by 20 publications

References 16 publications

Smart Intra-query Fault Tolerance for Massive Parallel Processing Databases

Smart Intra-query Fault Tolerance for Massive Parallel Processing Databases

Efficient fault-tolerance for iterative graph processing on distributed dataflow systems

An optimal checkpointing model with online OCI adjustment for stream processing applications

Contact Info

Product

Resources

About