Divide‐and‐conquer approach for solving singular value decomposition based on MapReduce

Abstract

Singular value decomposition (SVD) shows strong vitality in the area of information analysis and has significant application value in most of the scientific big data fields. However, with the rapid development of Internet, the information online reveals fast growing trend. For a large-scale matrix, applying SVD computation directly is both time consuming and memory demanding. There are many works available to speed up the computation of SVD based on the message passing interface model. However, to deal with largescale data processing, a MapReduce model has many advantages over a message passing interface model, such as fault tolerance, load balancing and simplicity. For a MapReduce environment, existing approaches only focus on low rank SVD approximation and tall-and-skinny matrix SVD computation, and there are no implementations of full rank SVD computation. In this paper, we propose a MapReduce-based implementation for solving divide-and-conquer SVD algorithm. To achieve high performance, we design a two-stage task scheduling strategy based on the mathematical characteristics of divide-and-conquer SVD algorithm. To further strengthen the performance, we propose a row-index-based divide algorithm, a pipelined task scheduling method, and revised block matrix multiplication in MapReduce framework. Experimental result shows the efficiency of our algorithm. Our implementation can accommodate full rank SVD computation of large-scale matrix very efficiently. 332 S. ZHAO ET AL. model. Hadoop relies on a distributed file system to share large-scale dataset among clusters. MapReduce procedure includes a shuffle phase where intermediate result is written on disk. Thus, we do not expect algorithms running in MapReduce framework to be faster than algorithms deployed in a state-of-the-art in-memory message passing interface(MPI) cluster. However, MapReduce model has many advantages over MPI model, which makes it attractive to process large-scale dataset. MapReduce model is designed to be fault tolerant while MPI model is not. Moreover, MapReduce model offers mechanisms to automatically deal with load balancing issues as well as input and output operations in a distributed environment, which greatly simplifies the programming. In today's information technology industry, many companies have employed Hadoop-distributed systems to deal with their daily business data.There are already implementations of SVD computation in MapReduce available. However, these implementations only focus on two aspects. The first one is low rank SVD approximation of matrices [3][4][5]. These research works are designed for those applications such as large-scale recommender systems in which only small fraction of singular values are needed. Nevertheless, for applications, such as matrix pseudo-inverse calculations that are widely used in large-scale medical and astronomy image processing field nowadays [6] and full rank principal component analysis in face recognition [7], performing a full rank SVD computation is required. Obviously, current implemen...

Advances in modelling and simulation for big‐data applications (AMSBA)

“…In , Zhao et al propose a Map‐Reduce‐based implementation for solving divide‐and‐conquer SVD algorithm, to exploit Big Data‐enabled architectures for high scale application of this fundamental and widely adopted computing tool, currently not covered by literature: this allows to benefit of inherent Map‐Reduce fault tolerance and load balancing features. The approach applies the intrinsic mathematical characteristics of the algorithm to deduce a two‐stage task‐scheduling strategy suited for the paradigm, achieving high efficiency by a pipeline‐oriented task‐scheduling design.…”