Executing parallel TwigStack algorithm on a multi-core system

Machdi, Imam; Amagasa, Toshiyuki; Kitagawa, Hiroyuki

doi:10.1145/1806338.1806376

Cited by 8 publications

(4 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several methods have been proposed for traditional parallel processing in multicore environments. Research in [9] proposes L2-cache optimizations in the traditional TwigStack platform to make it more efficient in multicore. Research in [7] adopts the traditional Metis graph partitioner for multicore and is capable of creating efficient parallel jobs from sequential assignments.…”

Section: Related Workmentioning

confidence: 99%

A software design and algorithms for multicore capture in data center forensics

Zhanikeev

2014

Proceedings of the 2nd International Workshop on Security and Forensics in Communication Systems

View full text Add to dashboard Cite

With rapid dissemination of cloud computing, data centers are quickly turning into platforms that host highly heterogeneous collections of services. Traditional approach to security and performance management finds it difficult to cope in such environments. Specifically, it is becoming increasingly difficult to capture and process all the necessary information at data centers in real time, where packet capture at data center gateways can serve as a practical example. This paper proposes a generic design for capturing and processing information on multicore architectures. The two main parts of the proposal are (1) the optimization formulation for distributing tasks across cores and (2) practical design and implementation of a shared memory which can be used for communication between processes in a non-traditional way that does not require memory locking or message passing.

show abstract

Section: Related Workmentioning

confidence: 99%

A software design and algorithms for multicore capture in data center forensics

Zhanikeev

2014

Proceedings of the 2nd International Workshop on Security and Forensics in Communication Systems

View full text Add to dashboard Cite

show abstract

“…Several methods have been proposed for traditional parallel processing in multicore environments. Previous research proposes L2‐cache optimizations in the traditional TwigStack platform to make it more efficient in multicore. Other research in adopts the traditional Metis graph partitioner for multicore and is capable of creating efficient parallel jobs from sequential assignments.…”

Section: Related Workmentioning

confidence: 99%

“…For example, this can be done through more efficient graph optimization . Low‐level CPU optimizations like those of L2‐cache is another practical solution for fast parallelization .…”

Section: Introductionmentioning

confidence: 99%

A lockfree shared memory design for high‐throughput multicore packet traffic capture

Zhanikeev

2014

Int J Network Mgmt

View full text Add to dashboard Cite

SUMMARYEarlier study by this author has shown that traffic can be captured in multicore architectures with negligible overhead, thus introducing the topic of multicore parallelization into packet traffic analysis. Designs that involve locking or message passing are considered default in parallel processing, which is incompatible with packet traffic with its high throughput and small grain. This paper proposes a new design for shared memory which can be used by any number of processes completely lockfree. Analysis of processing overhead and C/C + + implementation of the design are presented. Copyright © 2014 John Wiley & Sons, Ltd.

show abstract

“…In this section, we discuss particularly a data parallelism technique, which is fundamental to expose parallelism through the allocation of partitions over a number of processes (threads) associated with the available number of CPU cores (Machdi et al, 2009a).…”

Section: Data Parallelismmentioning

confidence: 99%

Parallel holistic twig joins on a multi‐core system

Machdi

Amagasa

Kitagawa

2010

International Journal of Web Information Systems

Self Cite

View full text Add to dashboard Cite

Purpose -The purpose of this paper is to propose general parallelism techniques for holistic twig join algorithms to process queries against Extensible Markup Language (XML) databases on a multi-core system. Design/methodology/approach -The parallelism techniques comprised data and task parallelism. As for data parallelism, the paper adopted the stream-based partitioning for XML to partition XML data as the basis of parallelism on multiple CPU cores. The XML data partitioning was performed in two levels. The first level was to create buckets for creating data independence and balancing loads among CPU cores; each bucket was assigned onto a CPU core. Within each bucket, the second level of XML data partitioning was performed to create finer partitions for providing finer parallelism. Each CPU core performed the holistic twig join algorithm on each finer partition of its own in parallel with other CPU cores. In task parallelism, the holistic twig join algorithm was decomposed into two main tasks, which were pipelined to create parallelism. The first task adopted the data parallelism technique and their outputs were transferred to the second task periodically. Since data transfers incurred overheads, the size of each data transfer needed to be estimated cautiously for achieving optimal performance. Findings -The data and task parallelism techniques contribute to good performance especially for queries having complex structures and/or higher values of query selectivity. The performance of data parallelism can be further improved by task parallelism. Significant performance improvement is attained by queries having higher selectivity because more outputs computed by the second task is performed in parallel with the first task.Research limitations/implications -The proposed parallelism techniques primarily deals with executing a single long-running query for intra-query parallelism, partitioning XML data on-the-fly, and allocating partitions on CPU cores statically. During the parallel execution, presumably there are no such dynamic XML data updates. Practical implications -The effectiveness of the proposed parallel holistic twig joins relies fundamentally on some system parameter values that can be obtained from a benchmark of the system platform. Originality/value -The paper proposes novel techniques to increase parallelism by combining techniques of data and task parallelism for achieving high performance. To the best of the author's knowledge, this is the first paper of parallelizing the holistic twig join algorithms on a multi-core system.

show abstract

Executing parallel TwigStack algorithm on a multi-core system

Cited by 8 publications

References 21 publications

A software design and algorithms for multicore capture in data center forensics

A software design and algorithms for multicore capture in data center forensics

A lockfree shared memory design for high‐throughput multicore packet traffic capture

Parallel holistic twig joins on a multi‐core system

Contact Info

Product

Resources

About