Integrating of Data Using the Hadoop and R

Uskenbayeva, Raissa; Kuandykov, A.A.; Cho, Young Im; Temirbolatova, T.; Amanzholova, Saule; Kozhamzharova, Dinara

doi:10.1016/j.procs.2015.07.187

Cited by 13 publications

(1 citation statement)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If yes then request is fulfilled by cache. If not it will be processes from filesystem and same will be copied to local cache memory for future reference (Liu & Dong, 2012;Uskenbayeva et al, 2015). Table 3 and figure 7 are showing the memory access time for file size 10 and using the associativity in the cache design.…”

Section: Methodsmentioning

confidence: 99%

Hadoop distributed file system mechanism for processing of large datasets across computers cluster using programming techniques

Edwards

Brain

Joly

et al. 2019

irjmis

View full text Add to dashboard Cite

In this paper, we have proved that the HDFS I/O operations performance is getting increased by integrating the set associativity in the cache design and changing the pipeline topology using fully connected digraph network topology. In read operation, since there is huge number of locations (words) at cache compared to direct mapping the chances of miss ratio is very low, hence reducing the swapping of the data between main memory and cache memory. This is increasing the memory I/O operations performance. In Write operation instead of using the sequential pipeline we need to construct the fully connected graph using the data blocks listed from the NameNode metadata. In sequential pipeline, the data is getting copied to source node in the pipeline. Source node will copy the data to next data block in the pipeline. The same copy process will continue until the last data block in the pipeline. The acknowledgment process has to follow the same process from last block to source block. The time required to transfer the data to all the data blocks in the pipeline and the acknowledgment process is almost 2n times to data copy time from one data block to another data block (if the replication factor is n).

show abstract

Section: Methodsmentioning

confidence: 99%