Research on Storage Strategy of Unstructured Small Files in HDFS

Abstract

With the wide use of HDFS and increasing scale of small files, problems of HDFS in small files storage gradually exposed. Thus the article put forward a storage strategy of unstructured small file based on the type of file, and optimized the architecture of cluster to save memory and improve the efficiency of file access. Through experiment, the strategy is proved to be effective and reliable.