Behavioral simulations in MapReduce

Wang, Guozhang; Salles, Marcos Vaz; Sowell, Benjamin; Wang, Xun; Cao, Tuan Vu; Demers, Alan; Gehrke, Johannes; White, Walker

doi:10.14778/1920841.1920962

Cited by 44 publications

(25 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We use a simple spatial partitioning method motivated by our distributed simulation platform [39]. While this method is unlikely to outperform special-purpose parallel join methods such as the TwinGrid index [32], it does not require modifying any of the join algorithms.…”

Section: Parallel Executionmentioning

confidence: 99%

“…As in [39], we address this by replicating all points within distance q of the boundary of each region, where q is the maximum radius of a query rectangle that makes up the join. Note that we do not replicate the query points in order to avoid over-counting.…”

Section: Parallel Executionmentioning

confidence: 99%

“…These include location-based services [28], games [40], virtual worlds [13], and scientific simulations, such as molecular dynamics [36] and behavioral simulations [39]. In these applications, moving objects continuously explore and sense their environment.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An experimental analysis of iterated spatial joins in main memory

Sowell¹,

Salles

Cao

et al. 2013

Proc. VLDB Endow.

Self Cite

View full text Add to dashboard Cite

Many modern applications rely on high-performance processing of spatial data. Examples include location-based services, games, virtual worlds, and scientific simulations such as molecular dynamics and behavioral simulations. These applications deal with large numbers of moving objects that continuously sense their environment, and their data access can often be abstracted as a repeated spatial join. Updates to object positions are interspersed with these join operations, and batched for performance. Even for the most demanding scenarios, the data involved in these joins fits comfortably in the main memory of a cluster of machines, and most applications run completely in main memory for performance reasons.Choosing appropriate spatial join algorithms is challenging due to the large number of techniques in the literature. In this paper, we perform an extensive evaluation of repeated spatial join algorithms for distance (range) queries in main memory. Our study is unique in breadth when compared to previous work: We implement, tune, and compare ten distinct algorithms on several workloads drawn from the simulation and spatial indexing literature. We explore the design space of both index nested loops algorithms and specialized join algorithms, as well as the use of moving object indices that can be incrementally maintained. Surprisingly, we find that when queries and updates can be batched, repeatedly re-computing the join result from scratch outperforms using a moving object index in all but the most extreme cases. This suggests that -given the code complexity of index structures for moving objects -specialized join strategies over simple index structures, such as Synchronous Traversal over R-Trees, should be the methods of choice for the above applications.

show abstract

Section: Parallel Executionmentioning

confidence: 99%

Section: Parallel Executionmentioning

confidence: 99%

See 1 more Smart Citation

An experimental analysis of iterated spatial joins in main memory

Sowell¹,

Salles

Cao

et al. 2013

Proc. VLDB Endow.

Self Cite

View full text Add to dashboard Cite

show abstract

“…In addition, a few animals try to influence the direction of movement of the whole group, e.g., because they have seen a predator or a food source. This simulation can be partitioned among multiple compute nodes through a spatial partitioning scheme [62]. At every time step of the simulation, neighboring nodes exchange messages before proceeding to the next time step.…”

Section: Goal: Time-to-solutionmentioning

confidence: 99%

ClouDiA: a deployment advisor for public clouds

et al. 2015

Self Cite

View full text Add to dashboard Cite

An increasing number of distributed data-driven applications are moving into shared public clouds. By sharing resources and operating at scale, public clouds promise higher utilization and lower costs than private clusters. To achieve high utilization, however, cloud providers inevitably allocate virtual machine instances non-contiguously; i.e., instances of a given application may end-up in physically distant machines in the cloud. This allocation strategy can lead to large differences in average latency between instances. For a large class of applications, this difference can result in significant performance degradation, unless care is taken in how application components are mapped to instances. In this paper, we propose ClouDiA, a general deployment advisor that selects application node deployments minimizing either (i) the largest latency between application nodes, or (ii) the longest critical path among all application nodes. ClouDiA employs a number of algorithmic techniques, including mixed-integer programming and constraint programming techniques, to efficiently search the space of possible mappings of application nodes to instances. Through experiments with synthetic and real applications in Amazon EC2, we show that mean latency is a robust metric to model

show abstract

“…Examples of such time-stepped applications include behavioral simulations [1], [2], [3], [4], and n-body problem [5], [6], [7], [8] and also are pervasive in graph algorithms [9], [10], [11], [12], scientific computing [13], [14] and so on. Recently, these applications have again attracted much attentions because they are becoming instrumental in characterizing physical, ecological, and societal systems.…”

Section: Introductionmentioning

confidence: 99%

Resisting Skew-Accumulation for Time-Stepped Applications in the Cloud via Exploiting Parallelism

Liao

Jin

Min

2015

IEEE Trans. Cloud Comput.

View full text Add to dashboard Cite

Abstract-Time-stepped applications are pervasive in scientific computing domain but perform poorly in the cloud because these applications execute in discrete time-step or tick and use logical synchronization barriers at tick boundaries to ensure correctness. As a result, the accumulated computational skew and communication skew that were unsolved in each tick can slow down time-stepped applications significantly. However, the existing solutions have focused only on the skew in each tick and thus cannot resist the accumulation of skew. To fill in this gap, an efficient approach to resisting the accumulation of skew is proposed in this paper via fully exploiting parallelism among ticks. This new approach allows the user to decompose much computational part (also called asynchronous part) of the processing for an object, into several asynchronous sub-processes which are dependent on one data object. Each sub-process from different ticks can then proceed in advance using the idle time whenever the needed data object is available, redressing the negative effects caused by accumulated unsolved computational and communication skew. To efficiently support such an approach, a data-centric programming model and also a runtime system, namely AsyTick, coupled with an ad hoc scheduler are developed. Experimental results show that the proposed approach can improve the performance of time-stepped applications over a state-of-the-art computational skew-resistant approach up to 2.53 times.

show abstract

Behavioral simulations in MapReduce

Cited by 44 publications

References 42 publications

An experimental analysis of iterated spatial joins in main memory

An experimental analysis of iterated spatial joins in main memory

ClouDiA: a deployment advisor for public clouds

Resisting Skew-Accumulation for Time-Stepped Applications in the Cloud via Exploiting Parallelism

Contact Info

Product

Resources

About