Filter decomposition for supporting coarse-grained pipelined parallelism

Abstract

We consider the filter decomposition problem in supporting coarse-grained pipelined parallelism. This form of parallelism is suitable for data-driven applications in scenarios where the data is available on a repository or a data collection site on the internet, and the final results are required on a user's desktop. A filter decomposition algorithm takes an application divided into a sequence of atomic filters, and maps them into a given number of filters.We propose three polynomial time algorithms for this problem. Dynamic programming algorithm MIN ONETRIP optimizes the one trip cost for a packet passing through the pipeline. MIN BOTTLENECK is also a dynamic programming algorithm, which minimizes the time spent on the bottleneck stage. Finally, MIN TOTAL is an approximate greedy algorithm which tries to minimize the total execution time.The results show that our heuristic algorithms work quite well in practice, with the possible exception of MIN ONETRIP when the number of packets is large. However, the relative performance of the algorithms is not always what we would expect, because of the certain limitations in how we model the problem.