BRPS: A Big Data Placement Strategy for Data Intensive Applications

Liu, Lihui; Song, Junping; Wang, Haibo; Lv, Pin

doi:10.1109/icdmw.2016.0120

Cited by 8 publications

(8 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If the individuals in the population are not stratified, crossover, mutation and fast non-dominated sort are executed in the initial population. Furthermore, according to (18) and Eq. (19), the crossover population H and the mutant population R are obtained.…”

Section: Methods Overviewmentioning

confidence: 99%

“…Besides, to ensure the stability and security of many applications and data in the cloud, it is also wise to maintain the load balance of nodes. Liu et al [18] proposed a duplicate placement approach, aiming to optimize data transmission efficiency, enhance the parallel placement performance and improve the load balance. Zhao et al [19] comprehensively investigated the data acquisition time and the load balance, and a data placement method.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

PDM: Privacy-Aware Deployment of Machine-Learning Applications for Industrial Cyber–Physical Cloud Systems

Yin

et al. 2021

IEEE Trans. Ind. Inf.

View full text Add to dashboard Cite

The cyber-physical cloud systems (CPCSs) release powerful capability in provisioning the complicated industrial services. Due to the advances of machine learning in attack detection, a wide range of machine-learning applications are involved in industrial CPCSs. However, how to ensure the implementation efficiency of these applications, and meanwhile avoid the privacy disclosure of the datasets due to data acquisition by different operators, remain challenging for the design of the CPCSs. To fill this gap, a privacy-aware deployment method, named PDM, is devised for hosting the machine-learning applications in the industrial CPCSs. In PDM, the machine-learning applications are partitioned as multiple computing tasks with certain execution order, like workflows. Specifically, the deployment problem is formulated as a multi-objective problem for improving the implementation performance and resource utility. Then the most balanced and optimal strategy is selected by leveraging an improved differential evolution technique. Finally, through comprehensive experiments and comparison analysis, PDM is fully evaluated.

show abstract

Section: Methods Overviewmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

PDM: Privacy-Aware Deployment of Machine-Learning Applications for Industrial Cyber–Physical Cloud Systems

Yin

et al. 2021

IEEE Trans. Ind. Inf.

View full text Add to dashboard Cite

show abstract

“…Given this, data replication management is considered one of the hot spot researches in large-scale distributed systems. The data replication strategies can be categorized into two types: static replication 12,13,14,15 and dynamic replication 26,27,28,29,30,31 . Static data replication strategies create and manage replicas manually and are incapable to be adjusted with the various changes especially in a large-scale Cloud systems.…”

Section: Related Workmentioning

confidence: 99%

Dynamic data replication and placement strategy in geographically distributed data centers

Bouhouch

Zbakh

Tadonki

2022

Concurrency and Computation

View full text Add to dashboard Cite

With the evolution of geographically distributed data centers in the Cloud Computing landscape along with the amount of data being processed in these data centers, which is growing at an exponential rate, processing massive data applications become an important topic. Since a given task may require many datasets for its execution and the datasets are spread over several different data centers, finding an efficient way to manage the datasets storage across nodes of a Cloud system is a difficult problem.In fact, the execution time of a task might be influenced by the cost of data transfers, which mainly depends on two criterias. The first one is the initial placement of the input datasets during the build-time phase, while the second is the replication of the datasets during the runtime phase. The replication is explicitly consider when datasets are being migrated over the data centers in order to make them locally available wherever needed. Data placement and data replication are important challenges in Cloud Computing. Nevertheless, many studies focus on data placement or data replication exclusively. In this paper, a combination of a data placement strategy followed by a dynamic data replication management strategy is proposed, with the purpose of reducing the associated cost of all data transfers between the (distant) data centers. Our proposed data placement approach considers the main characteristics of a data center such as storage capacity and read/write speeds to efficiently store the datasets, while our dynamic data replication management approach considers three parameters: the number of replicas in the system, the dependency between datasets and tasks and the storage capacity of data centers. The decision of when and whether to keep or to delete replicas is determined by the fulfillment of those three parameters. Our approach estimates the total execution time of the tasks as well as the monetary cost, considering the data transfers activity. Our experiments are conducted using Cloudsim simulator. The obtained results show that our proposed strategies produce an efficient data management by reducing the overheads of the data transfers, compared to both a data placement without replication (by 76%) and the selected data replication approach from Kouidri et al. (by 52%), and by improving the financial cost.

show abstract

“…At the same time, some scholars have conducted comprehensive research on these indicators. In [19], the authors proposed a BPRS big data copy placement strategy, which can reduce the data movement of each data centre and improve the load‐balancing problem.…”

Section: Related Workmentioning

confidence: 99%

“…As the number of meteorological workflows and data offloaded to cloud centre increases rapidly [13, 14], increasing the resource utilisation of active nodes in cloud centre is also being paid more and more attention [15, 16], and it has become an important indicator to measure the performance of placement method [17, 18]. In addition, with the improvement of the confidentiality of meteorological data, there are some privacy conflicts between meteorological data, so that these conflicting meteorological data should avoid being placed on the same or neighbouring storage nodes to ensure the security of these privacy data [19 –21]. Therefore, while improving the resource utilisation of nodes, the placement of conflict data is also receiving increasing attention.…”

Section: Introductionmentioning

confidence: 99%

Data privacy‐based coordinated placement method of workflows and data

Huang

Xue

Hu³

et al. 2020

IET cyber-phys. syst.

View full text Add to dashboard Cite

With the rapid development of data acquisition technology, many industries data already have the characteristics of big data and cloud technology has provided strong support for the storage and complex calculations of these massive data. The meteorological department established the cloud data centre based on the existing storage and computing resources and rearranged the historical data to reduce the historical data access time of applications. However, the placement of each workflow and input data also affects the average data access time, which in turn affects the computing efficiency of the cloud data centre. At the same time, because of the collaborative processing of multiple nodes, the resource utilisation of cloud data centre has also been paid more and more attention. In addition, with the increase of data security requirements, some privacy conflict data should avoid being placed on the same or neighbouring nodes. In response to this challenge, based on the fat-tree network topology, this study proposes a data privacy protection-based collaborative placement strategy of workflow and data to jointly optimise the average data access time, the average resource utilisation, and the data conflict degree. Finally, a large number of experimental evaluations and comparative analyses verify the efficiency of the proposed method. T m Total data access time of the task t m T avg AC average data access time for M tasks U i col resource utilisation of the ith compute node U j sto resource utilisation of the jth storage node U average resource utilisation of all nodes C data conflict degree for all conflicting data inp

show abstract

BRPS: A Big Data Placement Strategy for Data Intensive Applications

Cited by 8 publications

References 9 publications

PDM: Privacy-Aware Deployment of Machine-Learning Applications for Industrial Cyber–Physical Cloud Systems

PDM: Privacy-Aware Deployment of Machine-Learning Applications for Industrial Cyber–Physical Cloud Systems

Dynamic data replication and placement strategy in geographically distributed data centers

Data privacy‐based coordinated placement method of workflows and data

Contact Info

Product

Resources

About