Speedup your analytics

Lu, Jiaheng; Chen, Yuxing; Herodotou, Herodotos; Babu, Shivnath

doi:10.14778/3352063.3352112

Cited by 42 publications

(7 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [67], Lee et al improved the locality of network and storage I/O operations on many-core systems running Big Data applications using Apache Hadoop MapReduce. In [68], Lu et al discussed the importance of proper parameter settings in high-performance database systems for Big Data. In [69], Zhang et al defined a new benchmark and a new set of tools for benchmarking database systems for Big Data applications.…”

Section: Enabling Technologies For Big Datamentioning

confidence: 99%

Research Trends, Enabling Technologies and Application Areas for Big Data

Lundberg¹,

Grahn²

2022

Algorithms

View full text Add to dashboard Cite

The availability of large amounts of data in combination with Big Data analytics has transformed many application domains. In this paper, we provide insights into how the area has developed in the last decade. First, we identify seven major application areas and six groups of important enabling technologies for Big Data applications and systems. Then, using bibliometrics and an extensive literature review of more than 80 papers, we identify the most important research trends in these areas. In addition, our bibliometric analysis also includes trends in different geographical regions. Our results indicate that manufacturing and agriculture or forestry are the two application areas with the fastest growth. Furthermore, our bibliometric study shows that deep learning and edge or fog computing are the enabling technologies increasing the most. We believe that the data presented in this paper provide a good overview of the current research trends in Big Data and that this kind of information is very useful when setting strategic agendas for Big Data research.

show abstract

Section: Enabling Technologies For Big Datamentioning

confidence: 99%

Research Trends, Enabling Technologies and Application Areas for Big Data

Lundberg¹,

Grahn²

2022

Algorithms

View full text Add to dashboard Cite

show abstract

“…Spark is characterized by its in-memory computation and high expressiveness [91]. Based on these capabilities, Spark has become a natural choice to support two components of Big Data in iterative and reactive applications [92]. Regarding the speed, Spark has been reported having ten times faster than MR on disk-resident tasks and a hundred times faster for the memory-resident task [93].…”

Section: Bim-iot and Big-data Principlementioning

confidence: 99%

Investigating Approaches of Integrating BIM, IoT, and Facility Management for Renovating Existing Buildings: A Review

et al. 2021

View full text Add to dashboard Cite

The importance of building information is highly attached to the ability of conventional storing to provide professional analysis. The Internet of Things (IoT) and smart devices offer a vast amount of live data stored in heterogeneous repositories, and hence the need for smart methodologies to facilitate IoT–BIM integration is very crucial. The first step to better integrating IoT and Building Information Modeling (BIM) can be performed by implementing the Service-Oriented-Architecture (SOA) to combining software and other services by replacing the sematic information that was failed to display elements of indoor conditions. The other development is to create link that able to update static models towards real-time models using SOA approach. The existing approach relies on one-way interaction; however, developing two-way communication to mimic human cognitive has become very crucial. The high-tech approach requires highly involving Cloud computations to better connect IoT devices throughout Internet infrastructure. This approach is based on the integration of Building Information Modeling (BIM) with real-time data from IoT devices aiming at improving construction and operational efficiencies and to provide high-fidelity BIM models for numerous applications. The paper discusses challenges, limitations, and barriers that face BIM–IoT integration and simultaneously solves interoperability issues and Cloud computing. The paper provides a comprehensive review that explores and identifies common emerging areas of application and common design patterns of the traditional BIM-IoT integration followed by devising better methodologies to integrate IoT in BIM.

show abstract

“…With the native support from various cloud computing services, Spark-based big data analytics has been thriving in academia and industry. Nevertheless, managing the resources with proper configuration for Spark jobs remains challenging [21,37].…”

Section: Introductionmentioning

confidence: 99%

SimCost: cost-effective resource provision prediction and recommendation for spark workloads

Chen

Hoque

et al. 2023

Distrib Parallel Databases

Self Cite

View full text Add to dashboard Cite

Spark is one of the most popular big data analytical platforms. To save time, achieve high resource utilization, and remain cost-effective for Spark jobs, it is challenging but imperative for data scientists to configure suitable resource portions.In this paper, we investigate the proper parameter values that meet workloads’ performance requirements with minimized resource cost and resource utilization time. We propose SimCost, a simulation-based cost model, to predict the performance of jobs accurately. We achieve low-cost training by taking advantage of simulation framework, i.e., Monte Carlo simulation, which uses a small amount of data and resources to make a reliable prediction for larger datasets and clusters. Our method’s salient feature is that it allows us to invest low training costs while obtaining an accurate prediction. Through empirical experiments with 12 benchmark workloads, we show that the cost model yields less than 5% error on average prediction accuracy, and the recommendation achieves up to 6x resource cost saving.

show abstract

Speedup your analytics

Cited by 42 publications

References 21 publications

Research Trends, Enabling Technologies and Application Areas for Big Data

Research Trends, Enabling Technologies and Application Areas for Big Data

Investigating Approaches of Integrating BIM, IoT, and Facility Management for Renovating Existing Buildings: A Review

SimCost: cost-effective resource provision prediction and recommendation for spark workloads

Contact Info

Product

Resources

About