Big Data Architecture

Bhashyam, Ramesh

doi:10.1007/978-81-322-2494-5_2

Cited by 19 publications

(9 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Tableau is a software for interactive data visualization. [6], [9], [31], [47], [55], [68], [76], [82], [85] Apache Flume 7 [1], [6], [27], [52], [61], [70], [83] Apache Sqoop 5 [1], [27], [47], [52], [ [6], [21], [40], [41], [45] MongoDB 6 [16], [33], [35], [41], [43], [ [1], [6], [8], [9], [12], [18], [24], [27], [33], [35], [40], [42], [44], [47], [49], [51], [52], [55], [61], [68], [69], [71],…”

Section: Resultsmentioning

confidence: 99%

“…Initial Accepted Scopus 108 53 papers: [1]- [3], [5], [9], [10], [13]- [19], [23]- [29], [31]- [33], [37], [40], [45], [49], [50], [57], [60]- [66], [68], [70], [71], [73], [76]- [78], [81]- [84], [88], [90], [91], [93]- [95] Springer 222 20 papers: [4], [6], [12], [21], [30], [36], [38], [39], [41]- [43], [47], [51], [53], [69], [74], [79], [85], [86], [92] Google Scholar 197 6 papers:...…”

Section: Sourcementioning

confidence: 99%

See 1 more Smart Citation

A Mapping Study about Data Lakes: An Improved Definition and Possible Architectures

Couto¹,

Borges²,

Ruiz³

et al. 2019

International Conferences on Software Engineering and Knowledge Engineering

View full text Add to dashboard Cite

In the past few years, data lakes emerged as a trending topic in big data technologies. Although literature presents different points of view related to its functionalities, it serves mainly to store a variety of data in a big data context. In this paper, we aim to identify and analyze data lake definitions and possible architectures. Our methodology was composed of a systematic literature mapping based on PRISMA, software engineering best practices to perform reviews, and Kappa method to assess results' quality. We performed the search in eight different electronic databases to achieve a wide variety of publishers in Computer Science. We first identified 662 papers matching our search criteria; after filtering, we selected 87 papers for review. We found that the term data lakes was first defined by James Dixon in 2010. We also found that the term is often related to raw data repositories. From the identified definitions, we propose a new one as a means to better state what data lakes refer to and improve how the community use them. Moreover, we foind that Hadoop and its ecosystem compose the most used toolset to create data lakes, revealing that this is the mainstream in architectures for data lakes as of today's available technologies.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Sourcementioning

confidence: 99%

A Mapping Study about Data Lakes: An Improved Definition and Possible Architectures

Couto¹,

Borges²,

Ruiz³

et al. 2019

International Conferences on Software Engineering and Knowledge Engineering

View full text Add to dashboard Cite

show abstract

“…In the construction of the data center, it is often necessary to integrate data from multiple heterogeneous data sources [15][16][17]. These data are not the same even divergence greatly in many aspects, such as schemas, formats, forms, and so on.…”

Section: Discussionmentioning

confidence: 99%

Key Problems of Data Management in Data Center in Big Data Context

Lv¹,

Yan²

2017

dtetr

View full text Add to dashboard Cite

“…The range of four milling parameters is vc∈ [80,120], fz∈ [0.08,0.14], ae∈ [0.5,1.5] and ap∈ [8,12]. The random number m [-10,10].…”

Section: Source Of Datamentioning

confidence: 99%

“…Therefore, it is urgent to introduce new methods to deal with large-scale data calculation. Distributed framework is an effective way to solve the problem of massive data storage and computing, which makes big data processing possible [12,13]. Hadoop integrates Hdfs distributed storage [14] and MapReduce distributed computing framework [15], which originated from Apache Nutch project, started in 2002, and is one of the subprojects of Apache Lucene [16].…”

Section: Introductionmentioning

confidence: 99%

Research on Parallel Distributed Clustering Algorithm Applied to Milling Parameter Optimization

Wei

Sun

Liu

et al. 2021

Preprint

View full text Add to dashboard Cite

In the big data era, traditional data mining technology cannot meet the requirements of massive data processing with the background of intelligent manufacturing. Aiming at insufficient computing power and low efficiency in mining process, this paper proposes a improved K-means clustering algorithm based on the concept of distributed clustering in cloud computing environment. The improved algorithm (T.K-means) is combined with MapReduce computing framework of Hadoop platform to realize parallel computing, so as to perform processing tasks of massive data. In order to verify the practical performance of T.K-means algorithm, taking machining data of milling Ti-6Al-4V alloy as the mining object. The mapping relationship among milling parameters, surface roughness and material removal rate is mined, and the optimized value for milling parameters are obtained. The results show that T.K-means algorithm can be used to mine the optimal milling parameters, so that the best surface roughness can be obtained in milling Ti-6Al-4V titanium alloy.

show abstract

Big Data Architecture

Cited by 19 publications

References 6 publications

A Mapping Study about Data Lakes: An Improved Definition and Possible Architectures

A Mapping Study about Data Lakes: An Improved Definition and Possible Architectures

Key Problems of Data Management in Data Center in Big Data Context

Research on Parallel Distributed Clustering Algorithm Applied to Milling Parameter Optimization

Contact Info

Product

Resources

About