Parallel bulk insertion for large-scale analytics applications

Barbuzzi, Antonio; Michiardi, Pietro; Biersack, Ernst W.; Boggia, Gennaro

doi:10.1145/1859184.1859192

Cited by 11 publications

(6 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a consequence of which, all the cluster heads require the updated DHT status to guarantee correctness of results for fired queries. Typically, in distributed systems insertion is a recurrent and expensive operation [4]. Thus, a specialised design of bulk insertion is proposed.…”

Section: Insertionmentioning

confidence: 99%

Insertion and querying mechanism for a distributed XML database system

Kulkarni

Thirunavukkarasu

Pillai

et al. 2012

Proceedings of the 5th ACM COMPUTE Conference: Intelligent &Amp; Scalable System Technologies

View full text Add to dashboard Cite

XML files are ideal for describing semi structured data. The advent of native XML databases has enhanced storage of a large number of XML files preserving the hierarchical format. The challenge is to serve and access XML data from geographically distributed native XML databases. In this paper, we discuss the modus operandi for insertion and querying of data in a location transparent manner in decentralised networks. We present the design for a cluster based architecture where query processingF is achieved using distributed hash tables. A bulk insertion method has been employed for insertion of XML data. The proposed design was implemented on eXist-db, a native XML database engine. Experiments were conducted and the effectiveness of the proposed design was evaluated.

show abstract

Section: Insertionmentioning

confidence: 99%

Insertion and querying mechanism for a distributed XML database system

Kulkarni

Thirunavukkarasu

Pillai

et al. 2012

Proceedings of the 5th ACM COMPUTE Conference: Intelligent &Amp; Scalable System Technologies

View full text Add to dashboard Cite

show abstract

“…As such, there has been significant interest in developing high-performance parallel and concurrent algorithms and implementations of maps (e.g., see Section 2). Beyond simple insertion, deletion, and search, this work has considered "bulk" functions over ordered maps, such as unions [20,33,12], bulk-insertion and bulk-deletion [26,6,24], and range extraction [14,55,7].…”

Section: Introductionmentioning

confidence: 99%

PAM: Parallel Augmented Maps

Sun,

Ferizovic,

Blelloch

2016

Preprint

View full text Add to dashboard Cite

Ordered (key-value) maps are an important and widely-used data type for large-scale data processing frameworks. Beyond simple search, insertion and deletion, more advanced operations such as range extraction, filtering, and bulk updates form a critical part of these frameworks.We describe an interface for ordered maps that is augmented to support fast range queries and sums, and introduce a parallel and concurrent library called PAM (Parallel Augmented Maps) that implements the interface. The interface includes a wide variety of functions on augmented maps ranging from basic insertion and deletion to more interesting functions such as union, intersection, filtering, extracting ranges, splitting, and range-sums. We describe algorithms for these functions that are efficient both in theory and practice.As examples of the use of the interface and the performance of PAM we apply the library to four applications: simple range sums, interval trees, 2D range trees, and ranked word index searching. The interface greatly simplifies the implementation of these data structures over direct implementations. Sequentially the code achieves performance that matches or exceeds existing libraries designed specially for a single application, and in parallel our implementation gets speedups ranging from 40 to 90 on 72 cores with 2-way hyperthreading.

show abstract

“…Modern data analytics applications that operate on Big Data call for a parallel approach to data processing [1]. The most typical applications for large-scale data analysis are for data mining, web-page ranking, graph analysis, and model fitting [2].…”

Section: Introductionmentioning

confidence: 99%

A real-time stream storage and analysis platform for underwater acoustic monitoring

et al. 2013

View full text Add to dashboard Cite

We describe a distributed, real-time system for the collection and analysis of underwater acoustic data. The system uses a number of preprocessing steps to classify and detect acoustic events and to identify and compensate for gaps in the data stream. Different event-detection techniques are applied in a distributed manner on the incoming data stream from each sensor to aid in the indexing and storage of the data. Other event-detection techniques process multiple simultaneous streams to identify and classify events of interest. Building upon the deployed system, a stream analytical platform provides data handling, preprocessing, and analytics in real time. These analytics identify and classify anthropogenic, environmental, and animal noise (a significant amount of which occurs outside the audible range of human hearing) and ascertain the direction of the noise source.

show abstract

Parallel bulk insertion for large-scale analytics applications

Cited by 11 publications

References 7 publications

Insertion and querying mechanism for a distributed XML database system

Insertion and querying mechanism for a distributed XML database system

PAM: Parallel Augmented Maps

A real-time stream storage and analysis platform for underwater acoustic monitoring

Contact Info

Product

Resources

About