Principal component analysis (PCA) aims at estimating the direction of maximal variability of a high-dimensional dataset. A natural question is: does this task become easier, and estimation more accurate, when we exploit additional knowledge on the principal vector? We study the case in which the principal vector is known to lie in the positive orthant. Similar constraints arise in a number of applications, ranging from analysis of gene expression data to spike sorting in neural signal processing.In the unconstrained case, the estimation performances of PCA has been precisely characterized using random matrix theory, under a statistical model known as the 'spiked model.' It is known that the estimation error undergoes a phase transition as the signal-to-noise ratio crosses a certain threshold. Unfortunately, tools from random matrix theory have no bearing on the constrained problem. Despite this challenge, we develop an analogous characterization in the constrained case, within a one-spike model.In particular: (i) We prove that the estimation error undergoes a similar phase transition, albeit at a different threshold in signal-to-noise ratio that we determine exactly; (ii) We prove that -unlike in the unconstrained case-estimation error depends on the spike vector, and characterize the least favorable vectors; (iii) We show that a non-negative principal component can be approximately computed -under the spiked model-in nearly linear time. This despite the fact that the problem is non-convex and, in general, NP-hard to solve exactly.

The success of any user-generated content website depends crucially on its asset of content contributors. How firms should invest in the acquisition and retention of content contributors represents a novel question that is particularly important for these websites. We develop a vector autoregressive (VAR) model to measure the financial values of the retention and acquisition of both contributors and content consumers. In our empirical application to a customer-to-customer marketplace, we find that contributor (seller) acquisition has the largest financial value because of their strong network effects on content consumers (buyers) and other contributors. However, the wear-in of contributors' financial values takes longer because the network effects need time to be fully realized. Our simulation-based studies (i) shed light on the value implications of “enhancing network effects” and (ii) quantify the revenue contributions of marketing newsletter campaigns. Our results indicate that enhancing network effects in complementary ways can further increase the marginal benefits of acquisition and retention. We also find that simply tracking click-throughs may vastly underestimate the values of marketing newsletters—in our case, by more than a factor of 5—which may lead to suboptimal marketing effort allocation.

We consider the Principal Component Analysis problem for large tensors of arbitrary order k under a single-spike (or rank-one plus noise) model. On the one hand, we use information theory, and recent results in probability theory, to establish necessary and sufficient conditions under which the principal component can be estimated using unbounded computational resources. It turns out that this is possible as soon as the signal-to-noise ratio β becomes larger than C √ k log k (and in particular β can remain bounded as the problem dimensions increase).On the other hand, we analyze several polynomial-time estimation algorithms, based on tensor unfolding, power iteration and message passing ideas from graphical models. We show that, unless the signal-to-noise ratio diverges in the system dimensions, none of these approaches succeeds. This is possibly related to a fundamental limitation of computationally tractable estimators for this problem.We discuss various initializations for tensor power iteration, and show that a tractable initialization based on the spectrum of the matricized tensor outperforms significantly baseline methods, statistically and computationally. Finally, we consider the case in which additional side information is available about the unknown signal. We characterize the amount of side information that allows the iterative algorithms to converge to a good estimate.

No abstract

We consider the extragradient method to minimize the sum of two functions, the first one being smooth and the second being convex. Under the Kurdyka-Lojasiewicz assumption, we prove that the sequence produced by the extragradient method converges to a critical point of the problem and has finite length. The analysis is extended to the case when both functions are convex. We provide, in this case, a sublinear convergence rate, as for gradient-based methods. Furthermore, we show that the recent small-prox complexity result can be applied to this method. Considering the extragradient method is an occasion to describe an exact line search scheme for proximal decomposition methods. We provide details for the implementation of this scheme for the one norm regularized least squares problem and demonstrate numerical results which suggest that combining nonaccelerated methods with exact line search can be a competitive choice.