Distributed Matrix Completion

Exact and Approximate Maximum Inner Product Search with LEMP

“…In some applications, such as the ones above, MIPS is applied to the factor matrices obtained from some matrix factorization algorithm. Fast and scalable matrix factorization algorithms have been extensively studied in the literature [Makari et al 2015;Niu et al 2011;Teflioudi et al 2012] and the factorization itself is usually not a bottleneck (see Sec. 8.1 for some examples).…”

“…For Netflix, we performed a plain matrix factorization with DSGD++ using L2 regularization with regularization parameter λ = 50, as in [Teflioudi et al 2012]. For KDD, we used the factorization of Koenigstein et al [2011], 10 which incorporates the music taxonomy, temporal effects, as well as user and item biases; this dataset has been used in previous studies of the Top-k-MIPS problem.…”

Scaling data mining in massively parallel dataflow systems

“…al presented a parallel matrix factorization using Stochastic Gradient Descent [73], which leverages an intelligent partitioning to avoid conflicting updates, where each iteration only works on a subset of the data and leverages an intelligent partitioning that avoids conflicting updates. While their approach converges faster than ALS, they switched their implementation to MPI [167] due to the overhead incurred by Hadoop. Recht et al proposed a biased sampling approach to avoid conflicting updates during parallel training [145] and even proofed convergence under a minor amount of update conflicts [146].…”

Distributed matrix factorization with mapreduce using a series of broadcast-joins

“…Unfortunately, SGD is inherently sequential, because it updates the model parameters after each processed interaction. Techniques for parallel SGD have been proposed, yet they are either hard to implement, exhibit slow convergence or require shared-memory [11,16,20].…”

“…A prediction for the strength of the relation between a user and an item (e.g., the preference of a user towards a movie) is given by the dot product u ⊤ i mj of the vectors for user i and item j in the low-dimensional feature space. A popular technique to compute such a factorization is Stochastic Gradient Descent (SGD) [11,13,20], which randomly loops through all observed interactions aij, computes the error of the prediction u ⊤ i mj for each interaction and modifies the model parameters in the opposite direction of the gradient. Another technique is Alternating Least Squares (ALS) [12,23], which repeatedly keeps one of the unknown matrices (either U or M ) fixed, so that the other one can be optimally re-computed.…”