Convexity in Source Separation : Models, geometry, and algorithms

Abstract

Source separation or demixing is the process of extracting multiple components entangled within a signal. Contemporary signal processing presents a host of difficult source separation problems, from interference cancellation to background subtraction, blind deconvolution, and even dictionary learning. Despite the recent progress in each of these applications, advances in high-throughput sensor technology place demixing algorithms under pressure to accommodate extremely high-dimensional signals, separate an ever larger number of sources, and cope with more sophisticated signal and mixing models. These difficulties are exacerbated by the need for real-time action in automated decision-making systems.Recent advances in convex optimization provide a simple framework for efficiently solving numerous difficult demixing problems. This article provides an overview of the emerging field, explains the theory that governs the underlying procedures, and surveys algorithms that solve them efficiently. We aim to equip practitioners with a toolkit for constructing their own demixing algorithms that work, as well as concrete intuition for why they work. Fundamentals of demixingThe most basic model for mixed signals is a superposition model, where we observe a mixed signal z 0 ∈ R d of the formand we wish to determine the component signals x 0 and y 0 . This simple model appears in many guises. Sometimes, superimposed signals come from basic laws of nature. The amplitudes of electromagnetic waves, for example, sum together at a receiver, making the superposition model (1) common in wireless communications. Similarly, the additivity of sound waves makes superposition models natural in speech and audio processing.Other times, a superposition provides a useful, if not literally true, model for more complicated nonlinear phenomena. Images, for example, can be modeled as the sum of constituent featuresthink of stars and galaxies that sum to create an image of a piece of the night sky [1]. In machine learning, superpositions can describe hidden structure [2], while in statistics, superpositions can model gross corruptions to data [3]. These models also appear in texture repair [4], graph clustering [5], and line-spectral estimation [6].A conceptual understanding of demixing in all of these applications rests on two key ideas. Low-dimensional structures: Natural signals in high dimensions often cluster around lowdimensional structures with few degrees of freedom relative to the ambient dimension [7].Examples include bandlimited signals, array observations from seismic sources, and natural

A Survey on Nonconvex Regularization-Based Sparse and Low-Rank Recovery in Signal Processing, Statistics, and Machine Learning

“…More specifics on the applications involving the model (19) are as follows. 1) Source separation: such as the separation of texture in images [97], [98] and the separation of neuronal calcium transients in calcium imaging [99], A 1 and A 2 are two dictionaries allowing for sparse representation of the two distinct features, x 1 and x 2 are the (sparse or approximately sparse) coefficients describing these features [100]- [102]. 2) Super-resolution and inpainting: in the super-resolution and inpainting problem for images, audio, and video signals [103]- [105], only a subset of the desired signal y 0 = A 1 x 1 is available.…”

“…When both g 1 and g 2 are the 1 penalty, i.e., g 1 = g 2 = · 1 , (20) reduces to the sparse separation formulation in [102]. Further, when g 1 = g 2 = · 1 and µ = 1, the formulation (20) degenerates to the BP form considered in [100].…”

Convex Optimization for Big Data: Scalable, randomized, and parallel algorithms for big data analytics

“…First, (12) can address non-smooth and non-Lipschitz objective functions that commonly occur in many applications [13,17], such as robust principal component analysis (RPCA), graph learning, and Poisson imaging, in addition to the composite objectives we have covered so far. Second, we can apply a simple algorithm, called the alternating direction method of multipliers (ADMM) for its solutions, which leverages powerful augmented Lagrangian and dual decomposition techniques [4,18]:…”

Computational methods for underdetermined convolutive speech localization and separation via model-based sparse component analysis

“…The counting function in (11) is replaced with a sparsity inducing convex norm defined for the convex hull of the union of sparse vectors S. Therefore, a convex objective is obtained which can be solved using convex optimization (McCoy et al, 2014). We consider a particular extension of basis pursuit algorithm which relies on L 1 -norm (defined as the sum of absolute values of the vector elements) relaxation of the counting objective and L 1 L 2 formulation of group sparse recovery (Berg and Friedlander, 2008).…”