The processing of data-intensive workloads is a challenging and time-consuming task that often requires massive infrastructure to ensure fast data analysis. The cloud platform is the most popular and powerful scale-out infrastructure to perform big data analytics and eliminate the need to maintain expensive and high-end computing resources at the user side. The performance and the cost of such infrastructure depend on the overall server configuration, such as processor, memory, network, and storage configurations. In addition to the cost of owning or maintaining the hardware, the heterogeneity in the server configuration further expands the selection space, leading to non-convergence. The challenge is further exacerbated by the dependency of the application’s performance on the underlying hardware. Despite an increasing interest in resource provisioning, few works have been done to develop accurate and practical models to proactively predict the performance of data-intensive applications corresponding to the server configuration and provision a cost-optimal configuration online.
In this work, through a comprehensive real-system empirical analysis of performance, we address these challenges by introducing ProMLB: a proactive machine-learning-based methodology for resource provisioning. We first characterize diverse types of data-intensive workloads across different types of server architectures. The characterization aids in accurately capture applications’ behavior and train a model for prediction of their performance.
Then, ProMLB builds a set of cross-platform performance models for each application. Based on the developed predictive model, ProMLB uses an optimization technique to distinguish close-to-optimal configuration to minimize the product of execution time and cost. Compared to the oracle scheduler, ProMLB achieves 91% accuracy in terms of application-resource matching. On average, ProMLB improves the performance and resource utilization by 42.6% and 41.1%, respectively, compared to baseline scheduler. Moreover, ProMLB improves the performance per cost by 2.5× on average.
The widespread usage of internet, limited bandwidth of networks and different types of media all around the net causes a vast growth in compressing data with different abilities and qualities. Nowadays, video is a popular media for everyday usage. In different research areas, there is a need for recording events in high frame rates. Due to the high frame rate video constraints, using complex methods are not suitable for real-time coding of these videos and will increase the cost of the system. There are different lossless, lossy and near-lossless methods for compressing video sequences. Existing lossy methods cannot limit the subjective or objective loss to a certain upper bound. There have been works regarding lossless compression of these sequences, however these works offer modest compression ratios and in some cases will not be enough due to the large size of these sequences. In this paper we propose a near-lossless method that is comparable with successful existing methods of video compression and yet is simple enough for real-time applications. It includes the major conventional parts for this goal which are prediction, quantization and entropy coding. A simple rate control is embedded by different approaches in quantization. The experimental results demonstrate good compression ratios while considering reliability due to control of the maximum pixel error.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.