Semantic characterization of MapReduce workloads

Abstract

Abstract-MapReduce is a platform for analyzing large amounts of data on clusters of commodity machines. MapReduce is popular, in part thanks to its apparent simplicity. However, there are unstated requirements for the semantics of MapReduce applications that can affect their correctness and performance. MapReduce implementations do not check whether user code satisfies these requirements, leading to time-consuming debugging sessions, performance problems, and, worst of all, silently corrupt results. This paper makes these requirements explicit, framing them as semantic properties and assumed outcomes. It describes a black-box approach for testing for these properties, and uses the approach to characterize the semantics of 23 non-trivial MapReduce workloads. Surprisingly, we found that for most requirements, there is at least one workload that violates it. This means that MapReduce may be simple to use, but it is not as simple to use correctly. Based on our results, we provide insights to users on how to write higher-quality MapReduce code, and insights to system and language designers on ways to make their platforms more robust.

Nondeterminism in MapReduce considered harmful? an empirical study on non-commutative aggregators in MapReduce programs

“…Our study shows that certain data properties (functional dependencies) in the data can render non-commutative reducers harmless, making a case for checking data properties. In addition, 11 out of 23 reducers in [13] are non-commutative and exhibit the StrConcat pattern according to their study. The distribution of non-commutativity patterns is not consistent with our results, probably because open-sourced MapReduce programs in their study process simpler data with fewer cross-column relations than our samples from the production environment as discussed in Section 3.2.…”

“…Csallner et al [3] proposed a white-box symbolic execution approach to test commutativity. Xu et al [13] studied 23 reducers in open-sourced MapReduce programs and used a blackbox approach to test commutativity. Both of them check code and generate different input sequences with the same data set, on which the target reducer outputs different results.…”

Testing properties of dataflow program operators

“…In follow-up work, we adapted our approach to MapReduce, and applied it to characterize semantic properties of MapReduce workloads [25]. As future work, we intend to consider the next stages of dataflow program validation, in which testing approaches analogous to integration and system testing are needed.…”

An Executable Sequential Specification for Spark Aggregation

“…Verifying the correctness of a MR program involves checking the commutativity and associativity of the reduce function. Xu et al propose various semantic criteria to model commonly held assumptions on MR programs [29], including determinism, partition isolation, commutativity, and associativity of map/reduce combinators. Their empirical survey shows that these criteria are often overlooked by programmers and violated in practice.…”