TurBO: A cost-efficient configuration-based auto-tuning approach for cluster-based big data frameworks

Dou, Hui; Zhang, Lei; Zhang, Yiwen; Chen, Pengfei; Zheng, Zibin

doi:10.1016/j.jpdc.2023.03.002

Cited by 3 publications

(3 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Before recommending the target user, the neighbor set N of the user U is found and represented by S (u, I), the book types preferred by the user in S are listed, and the book types in each list are quantified through a formula [10]. O as to calculate the preference degree of the user for the book type I, sort the preference degree of the user for each type of book from high to low, and recommend the book with high preference degree to the target user.…”

Section: Precise Recommendation Service Implementation Processmentioning

confidence: 99%

Research on Library Accurate Recommendation System Based on Big Data Technology and User Portrait

Duan

2024

Proceedings of the First International Conference on Science, Engineering and Technology Practices for Sustainable Development,

View full text Add to dashboard Cite

As an information sharing center, university library covers almost all languages and various carriers of information, so how to effectively use information resources and give full play to the service role of the library is very important. Nowadays, with the rapid development of the Internet, big data and cloud computing, the undifferentiated recommended content of the existing library management system has been unable to meet the diverse and personalized needs of users; for a large number of user data accumulated in the library management system over the years, the value of data needs to be excavated. Therefore, the use of data technology for innovation has become a new driving force for the development and transformation of libraries. User portrait is a kind of data analysis tool which can process the data related to users, extract the user feature vector, and then get the user feature model to display the user panorama intuitively. This paper applies the user portrait to the library field, constructs the portrait model through data analysis, and uses the portrait to effectively predict user preferences, user interests and user behavior, which can be used as a basis to achieve accurate recommendation of library books and meet the personalized needs of readers.

show abstract

Section: Precise Recommendation Service Implementation Processmentioning

confidence: 99%

Research on Library Accurate Recommendation System Based on Big Data Technology and User Portrait

Duan

2024

Proceedings of the First International Conference on Science, Engineering and Technology Practices for Sustainable Development,

View full text Add to dashboard Cite

show abstract

“…Several optimizations methods [ 24 , 25 , 26 , 27 , 28 ] are discussed to improve the performance of distributed systems. Donta et al [ 24 ] summarize various message queues and message brokers used in IoT systems, and they find out that multiple message queues handle messages as per predefined constraints, making them static in nature.…”

Section: Related Workmentioning

confidence: 99%

“…Experimental results conducted on a local Spark cluster with HiBench benchmark applications showcase the effectiveness of DeepCAT in achieving improved performance with reduced tuning costs. Recently, Dou et al [ 28 ] propose a cost-efficient approach called TurBO that enhances Bayesian optimization (BO) to handle sub-optimal configurations for big data-processing frameworks. Their experimental evaluations on a local Spark cluster demonstrate that TurBO outperforms three representative baseline approaches, achieving significant speedup in the tuning process.…”

Section: Related Workmentioning

confidence: 99%

Towards an Optimized Distributed Message Queue System for AIoT Edge Computing: A Reinforcement Learning Approach

Xie

et al. 2023

Sensors

View full text Add to dashboard Cite

The convergence of artificial intelligence and the Internet of Things (IoT) has made remarkable strides in the realm of industry. In the context of AIoT edge computing, where IoT devices collect data from diverse sources and send them for real-time processing at edge servers, existing message queue systems face challenges in adapting to changing system conditions, such as fluctuations in the number of devices, message size, and frequency. This necessitates the development of an approach that can effectively decouple message processing and handle workload variations in the AIoT computing environment. This study presents a distributed message system for AIoT edge computing, specifically designed to address the challenges associated with message ordering in such environments. The system incorporates a novel partition selection algorithm (PSA) to ensure message order, balance the load among broker clusters, and enhance the availability of subscribable messages from AIoT edge devices. Furthermore, this study proposes the distributed message system configuration optimization algorithm (DMSCO), based on DDPG, to optimize the performance of the distributed message system. Experimental evaluations demonstrate that, compared to the genetic algorithm and random searching, the DMSCO algorithm can provide a significant improvement in system throughput to meet the specific demands of high-concurrency AIoT edge computing applications.

show abstract

CTuner: Automatic NoSQL Database Tuning with Causal Reinforcement Learning

Mai,

He,

et al. 2024

Proceedings of the 15th Asia-Pacific Symposium on Internetware

View full text Add to dashboard Cite

TurBO: A cost-efficient configuration-based auto-tuning approach for cluster-based big data frameworks

Cited by 3 publications

References 29 publications

Research on Library Accurate Recommendation System Based on Big Data Technology and User Portrait

Research on Library Accurate Recommendation System Based on Big Data Technology and User Portrait

Towards an Optimized Distributed Message Queue System for AIoT Edge Computing: A Reinforcement Learning Approach

CTuner: Automatic NoSQL Database Tuning with Causal Reinforcement Learning

Contact Info

Product

Resources

About