High Scalability of HDFS using Distributed Namespace

Abstract

In data intensive computing, Hadoop is widely used by organizations. The client applications of Hadoop require high availability and scalability of the system. Mostly, these applications are online and their data growth rate is unpredictable. The present Hadoop relies on secondary namenode for failover which slows down the performance of the system. Hadoop system's scalability depends on the vertical scalability of namenode server. As the namespace of Hadoop distributed file system grows, it demands additional memory to cache. A namenode server does not have enough primary memory to cache the namespace, its performance and availability effects. A new Hadoop architecture has been proposed to address the issues of namenode scalability, single point of failure and availability of Hadoop. This approach is based on distribution of namespace using distributed hash tables. The growing size of namespace of HDFS is distributed into multiple name node servers. The proposed architecture of Hadoop is simulated by using the multiple name node servers. The name node are arranges in chord ring. This allows HDFS to scale up horizontally. The system provides decartelize managed approach for namespace distribution which gives consistent performance. The results of HDFS namespace to store 1 billion or above files are discussed in this research work. The proposed architecture has shown high availability and adapts to name node failure. General TermsData intensive computing, Scalability, Failover, Availability

Towards the design of a film-based graphical password scheme

“…This makes it unnecessary to manage too many nodes, and therefore avoid too much overhead [10], [11]. Our architecture adopts the triple-machine Paxos algorithm for metadata consistency.…”

Smart Sensor-based Interface Controlin the Mobile CloudEnvironment

“…Hadoop distributed file system is copied to the storage and management of data to multiple, because even if the data loss can recover quickly. When you save the data to be stored in HDFS [11,12], Hadoop's Map when you read the information and reduce the ability to quickly analyze the data and search interfaces that are produced [13,14]. User information and interface as the value of the Key and Value data information sharing.…”