No abstract

No abstract

In this work, we present a set of techniques that considerably improve the performance of executing concurrent MapReduce jobs. Our proposed solution relies on proper resource allocation for concurrent Hive jobs based on data dependency, inter-query optimization and modeling of Hadoop cluster load. To the best of our knowledge, this is the first work towards Hive/MapReduce job optimization which takes Hadoop cluster load into consideration.We perform an experimental study that demonstrates 233% reduction in execution time for concurrent vs sequential execution schema. We report up to 40% extra reduction in execution time for concurrent job execution after resource usage optimization.The results reported in this paper were obtained in a pilot project to assess the feasibility of migrating A/B testing from Teradata + SAS analytics infrastructure to Hadoop. This work was performed on eBay production Hadoop cluster.

In this paper, we explore the feasibility of long-term prediction of buyer behavior using context-based approach based on variable-length markov chains. We discuss different strategies on event log pre-processing and the impact of this operation on the accuracy of the result. We report our results on the accuracy of the prediction of buyer / seller behavior on eBay marketplace for one week in the future 1 .

No abstract