Abstract-A mean-square error lower bound for the discretetime nonlinear filtering problem is derived based on the Van Trees (posterior) version of the Cramér-Rao inequality. This lower bound is applicable to multidimensional nonlinear, possibly non-Gaussian, dynamical systems and is more general than the previous bounds in the literature. The case of singular conditional distribution of the one-step-ahead state vector given the present state is considered. The bound is evaluated for three important examples: the recursive estimation of slowly varying parameters of an autoregressive process, tracking a slowly varying frequency of a single cisoid in noise, and tracking parameters of a sinusoidal frequency with sinusoidal phase modulation.

We propose a new low-complexity approximate joint diagonalization (AJD) algorithm, which incorporates nontrivial block-diagonal weight matrices into a weighted least-squares (WLS) AJD criterion. Often in blind source separation (BSS), when the sources are nearly separated, the optimal weight matrix for WLS-based AJD takes a (nearly) block-diagonal form. Based on this observation, we show how the new algorithm can be utilized in an iteratively reweighted separation scheme, thereby giving rise to fast implementation of asymptotically optimal BSS algorithms in various scenarios. In particular, we consider three specific (yet common) scenarios, involving stationary or block-stationary Gaussian sources, for which the optimal weight matrices can be readily estimated from the sample covariance matrices (which are also the target-matrices for the AJD). Comparative simulation results demonstrate the advantages in both speed and accuracy, as well as compliance with the theoretically predicted asymptotic optimality of the resulting BSS algorithms based on the weighted AJD, both on large scale problems with matrices of the size 100 2 100.Index Terms-Approximate joint diagonalization (AJD), autoregressive processes, blind source separation (BSS), nonstationary random processes.

CANDECOMP/PARAFAC (CP) has found numerous applications in wide variety of areas such as in chemometrics, telecommunication, data mining, neuroscience, separated representations. For an order-tensor, most CP algorithms can be computationally demanding due to computation of gradients which are related to products between tensor unfoldings and Khatri-Rao products of all factor matrices except one. These products have the largest workload in most CP algorithms. In this paper, we propose a fast method to deal with this issue. The method also reduces the extra memory requirements of CP algorithms. As a result, we can accelerate the standard alternating CP algorithms 20-30 times for order-5 and order-6 tensors, and even higher ratios can be obtained for higher order tensors (e.g.,). The proposed method is more efficient than the state-of-the-art ALS algorithm which operates two modes at a time (ALSo2) in the Eigenvector PLS toolbox, especially for tensors with order and high rank.

We propose an improved version of algorithm FastICA which is asymptotically efficient, i.e., its accuracy attains the Cramér-Rao lower bound provided that the probability distribution of the signal components belongs to the class of generalized Gaussian distribution. Its computational complexity is only slightly (about three times) higher than that of ordinary symmetric FastICA. Simulation section shows superior performance of the algorithm compared with JADE, and of non-parametric ICA.

No abstract

No abstract

Abstract. The damped Gauss-Newton (dGN) algorithm for CANDECOMP/PARAFAC (CP) decomposition can handle the challenges of collinearity of factors and different magnitudes of factors; nevertheless, for factorization of an N-D tensor of size I 1 × . . . × I N with rank R, the algorithm is computationally demanding due to construction of large approximate Hessian of size (RT × RT ) and its inversion where T = n I n . In this paper, we propose a fast implementation of the dGN algorithm which is based on novel expressions of the inverse approximate Hessian in block form. The new implementation has lower computational complexity, besides computation of the gradient (this part is common to both methods), requiring the inversion of a matrix of size NR 2 × NR 2 , which is much smaller than the whole approximate Hessian, if T ≫ NR. In addition, the implementation has lower memory requirements, because neither the Hessian nor its inverse never need to be stored in their entirety. A variant of the algorithm working with complex valued data is proposed as well. Complexity and performance of the proposed algorithm is compared with those of dGN and ALS with line search on examples of difficult benchmark tensors.

We revise the problem of extracting one independent component from an instantaneous linear mixture of signals. The mixing matrix is parameterized by two vectors, one column of the mixing matrix and one row of the de-mixing matrix. The separation is based on the non-Gaussianity of the source of interest, while the other background signals are assumed to be Gaussian. Three gradient-based estimation algorithms are derived using the maximum likelihood principle and are compared with the Natural Gradient algorithm for Independent Component Analysis and with One-unit FastICA based on negentropy maximization. The ideas and algorithms are also generalized for the extraction of a vector component when the extraction proceeds jointly from a set of instantaneous mixtures. Throughout the paper, we address the problem of the size of the region of convergence for which the algorithms guarantee the extraction of the desired source. We show how that size is influenced by the ratio of powers of the sources within the mixture. Simulations confirm this observation where several algorithms are compared. They show various convergence behaviour in a scenario where the source of interest is dominant or weak. Here, our proposed modifications of the gradient methods taking into account the dominance/weakness of the source show improved global convergence property.