SD2DS-Based Datastore for Large Files

Krechowicz, Adam; Chrobot, Arkadiusz; Deniziak, Stanislaw; Łukawski, Grzegorz

doi:10.1007/978-3-319-46535-7_13

Cited by 6 publications

(4 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As previously presented in the [22,25], SD2DS system already proved its value for processing Big Data sets. In those works the extensive evaluation of SD2DS with comparison to other Big Data storages (MongoDB, MemCached) was presented.…”

Section: Resultsmentioning

confidence: 67%

See 1 more Smart Citation

Content-aware data distribution over cluster nodes

Krechowicz¹

2021

IDA

Self Cite

View full text Add to dashboard Cite

Proper data items distribution may seriously improve the performance of data processing in distributed environment. However, typical datastorage systems as well as distributed computational frameworks do not pay special attention to that aspect. In this paper author introduces two custom data items addressing methods for distributed datastorage on the example of Scalable Distributed Two-Layer Datastore. The basic idea of those methods is to preserve that data items stored on the same cluster node are similar to each other following concepts of data clustering. Still, most of the data clustering mechanisms have serious problem with data scalability which is a severe limitation in Big Data applications. The proposed methods allow to efficiently distribute data set over a set of buckets. As it was shown by the experimental results, all proposed methods generate good results efficiently in comparison to traditional clustering techniques like k-means, agglomerative and birch clustering. Distributed environment experiments shown that proper data distribution can seriously improve the effectiveness of Big Data processing.

show abstract

Section: Resultsmentioning

confidence: 67%

“…Scalable Distributed Two Layer Datastore (SD2DS) [22] proved to be a very efficient solution for storing massive data sets. The basic idea of the SD2DS is the division of the data item (called component) into two separate parts: the component header (identified by a unique key k) and the component body.…”

Section: Scalable Distributed Two Layer Datastorementioning

confidence: 99%

Content-aware data distribution over cluster nodes

Krechowicz¹

2021

IDA

Self Cite

View full text Add to dashboard Cite

show abstract

“…The Scalable Distributed Two-layered Data Structures (SD2DS) [54] were introduced to overcome the disadvantages of single-layered structures. They allowed to develop an efficient datastore for large files [55], [56]. Apart from improving the split performance, they also allow to introduce other features like throughput scalability [57], [58] or anonymity [59], [60].…”

Section: Scalable Distributed Two-layered Datastorementioning

confidence: 99%

Highly Scalable Distributed Architecture for NoSQL Datastore Supporting Strong Consistency

2021

Self Cite

View full text Add to dashboard Cite

Introducing a strong consistency model into NoSQL data storages is one of the most interesting issues nowadays. In spite of the CAP theorem, many NoSQL systems try to strengthen the consistency to their limits to better serve the business needs. However, many consistency-related problems that occur in popular data storages are impossible to overcome and enforce rebuilding the whole system from scratch. Additionally, providing scalability to those systems really complicates the matter. In this paper, a novel data storage architecture that supports strong consistency without loosing scalability is proposed. It provides strong consistency according to the following requirements: high scalability, high availability, and high throughput. The proposed solution is based on the Scalable Distributed Two-Layer Data Store which has proven to be a very efficient NoSQL system. The proposed architecture takes into account the concurrent execution of operations and unfinished operations. The theoretical correctness of the architecture as well as experimental evaluation in comparison to traditional mechanisms like locking and versioning is also shown. Comparative analysis with popular NoSQL systems like MongoDB and MemCached is also presented. Obtained results show that the proposed architecture presents a very high performance in comparison to existing NoSQL systems.

show abstract

“…3 present the time of accessing the data in SD2DS in comparison to the most recognizable representatives of the NoSQL systems: MongoDB [17] and MemCached [18]. The overall evaluation was presented in [1] and [19]. The MongoDB and MemCached was chosen because they have many similarities with SD2DS.…”

Section: Scalable Distributed Two-layer Data Storementioning

confidence: 99%

Business Intelligence Platform for Big Data based on Scalable Distributed Two-Layer Data Store

Krechowicz¹,

Deniziak²

2017

Annals of Computer Science and Information Systems

Self Cite

View full text Add to dashboard Cite

Abstract-Data mining is one of the main business intelligence technique. The volume of Big Data expands in such a way that the classical Business Intelligence methods need to be redefined. First, the organization of large data sets requires new distributed database architectures. Second, it is necessary to develop distributed data processing models that provide a high degree of scalability.In this paper we introduce fully scalable BI platform that is suitable for the most common data processing issues. The platform is based on our scalable distributed two-layer data store, which is competitive to existing NoSQL distributed data base systems. We show examples and experimental results showing advantages of our approach.

show abstract

SD2DS-Based Datastore for Large Files

Cited by 6 publications

References 11 publications

Content-aware data distribution over cluster nodes

Content-aware data distribution over cluster nodes

Highly Scalable Distributed Architecture for NoSQL Datastore Supporting Strong Consistency

Business Intelligence Platform for Big Data based on Scalable Distributed Two-Layer Data Store

Contact Info

Product

Resources

About