We propose a general algorithmic framework for constrained matrix and tensor factorization, which is widely used in signal processing and machine learning. The new framework is a hybrid between alternating optimization (AO) and the alternating direction method of multipliers (ADMM): each matrix factor is updated in turn, using ADMM, hence the name AO-ADMM. This combination can naturally accommodate a great variety of constraints on the factor matrices, and almost all possible loss measures for the fitting. Computation caching and warm start strategies are used to ensure that each update is evaluated efficiently, while the outer AO framework exploits recent developments in block coordinate descent (BCD)-type methods which help ensure that every limit point is a stationary point, as well as faster and more robust convergence in practice. Three special cases are studied in detail: non-negative matrix/tensor factorization, constrained matrix/tensor completion, and dictionary learning. Extensive simulations and experiments with real data are used to showcase the effectiveness and broad applicability of the proposed framework.
Abstract-The Akaike information criterion (AIC) and the minimum description length (MDL) are two well-known criteria for model order selection in the additive white noise case. Our aim is to study the influence on their behavior of a large gap between the signal and the noise eigenvalues and of the noise eigenvalue dispersion. Our results are mostly qualitative and serve to explain the behavior of the AIC and the MDL in some cases of great practical importance. We show that when the noise eigenvalues are not clustered sufficiently closely, then the AIC and the MDL may lead to overmodeling by ignoring an arbitrarily large gap between the signal and the noise eigenvalues. For fixed number of data samples, overmodeling becomes more likely for increasing the dispersion of the noise eigenvalues. For fixed dispersion, overmodeling becomes more likely for increasing the number of data samples. Undermodeling may happen in the cases where the signal and the noise eigenvalues are not well separated and the noise eigenvalues are clustered sufficiently closely. We illustrate our results by using simulations from the effective channel order determination area.Index Terms-Akaike information criterion, minimum description length criterion.
Tensor factorization has proven useful in a wide range of applications, from sensor array processing to communications, speech and audio signal processing, and machine learning. With few recent exceptions, all tensor factorization algorithms were originally developed for centralized, in-memory computation on a single machine; and the few that break away from this mold do not easily incorporate practically important constraints, such as nonnegativity. A new constrained tensor factorization framework is proposed in this paper, building upon the Alternating Direction Method of Multipliers (ADMoM). It is shown that this simplifies computations, bypassing the need to solve constrained optimization problems in each iteration; and it naturally leads to distributed algorithms suitable for parallel implementation. This opens the door for many emerging big data-enabled applications. The methodology is exemplified using non-negativity as a baseline constraint, but the proposed framework can incorporate many other types of constraints. Numerical experiments are encouraging, indicating that ADMoM-based non-negative tensor factorization (NTF) has high potential as an alternative to state-of-the-art approaches.
The maximization of a full-rank quadratic form over a finite alphabet is NP-hard in both a worst-case sense and an average sense. Interestingly, if the rank of the form is not a function of the problem size, then it can be maximized in polynomial time. An algorithm for the efficient computation of the binary vector that maximizes a rank-deficient quadratic form is developed based on an analytic procedure. Auxiliary spherical coordinates are introduced and the multi-dimensional space is partitioned into a polynomial-size set of regions; each region corresponds to a distinct binary vector. The binary vector that maximizes the rank-deficient quadratic form is shown to belong to the polynomial-size set of candidate vectors. Thus, the size of the feasible set is efficiently reduced from exponential to polynomial.
The least-squares and the subspace methods are two well-known approaches for blind channel identification/ equalization. When the order of the channel is known, the algorithms are able to identify the channel, under the so-called length and zero conditions. Furthermore, in the noiseless case, the channel can be perfectly equalized. Less is known about the performance of these algorithms in the practically inevitable cases in which the channel possesses long tails of "small" impulse response terms. We study the performance of the m m mth-order least-squares and subspace methods using a perturbation analysis approach. We partition the true impulse response into the m m mth-order significant part and the tails. We show that the m m mth-order least-squares or subspace methods estimate an impulse response that is "close" to the m m mth-order significant part. The closeness depends on the diversity of the m m mth-order significant part and the size of the tails. Furthermore, we show that if we try to model not only the "large" terms but also some "small" ones, then the quality of our estimate may degrade dramatically; thus, we should avoid modeling "small" terms. Finally, we present simulations using measured microwave radio channels, highlighting potential advantages and shortcomings of the least-squares and subspace methods.
A novel method for the detection of ischaemic episodes in long duration ECGs is proposed. It includes noise handling, feature extraction, rule-based beat classification, sliding window classification and ischaemic episode identification, all integrated in a four-stage procedure. It can be executed in real time and is able to provide explanations for the diagnostic decisions obtained. The method was tested on the ESC ST-T database and high scores were obtained for both sensitivity and positive predictive accuracy (93.8% and 78.5% respectively using aggregate gross statistics, and 90.7% and 80.7% using aggregate average statistics).
Abstract-We consider minimum mean-square error Tomlinson-Harashima (MMSE-TH) precoding for time-varying frequency-selective channels. We assume that the receiver estimates the channel and sends the channel state information (CSI) estimate to the transmitter through a lossless feedback channel that introduces a certain delay. Thus, the CSI mismatch at the receiver is due to estimation errors, while the CSI mismatch at the transmitter is due to both estimation errors and channel time variations. We exploit a priori statistical channel knowledge, and we derive an optimal TH precoder, adopting a Bayesian approach. We use simulations to compare the performance of the so-derived TH precoder with that of the same-complexity MMSE decision-feedback equalizer (DFE). We observe that for low signal-to-noise ratios (SNRs) and sufficiently slow channel time variations, the optimal TH precoder outperforms the DFE, while at high SNR, the opposite happens.Index Terms-Intersymbol interference (ISI), partial channel knowledge, Rayeigh fading, Tomlinson-Harashima (TH) precoder.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.