Comparing Measures of Sparsity

Abstract

Sparsity of representations of signals has been shown to be a key concept of fundamental importance in fields such as blind source separation, compression, sampling and signal analysis. The aim of this paper is to compare several commonlyused sparsity measures based on intuitive attributes. Intuitively, a sparse representation is one in which a small number of coefficients contain a large proportion of the energy. In this paper six properties are discussed: (Robin Hood, Scaling, Rising Tide, Cloning, Bill Gates and Babies), each of which a sparsity measure should have. The main contributions of this paper are the proofs and the associated summary table which classify commonly-used sparsity measures based on whether or not they satisfy these six propositions. Only one of these measures satisfies all six: the Gini Index.

Single-Unit Recordings in the Macaque Face Patch System Reveal Limitations of fMRI MVPA

“…Sparseness reflects the proportion of a neural population that is active in response to a stimulus. We used the Gini index (Hurley and Rickard, 2009) as a measure of sparseness: if only one neuron in a population responds to a given stimulus, the Gini index is 1, whereas if all neurons respond at the same level (compared with their maximal response), the Gini index is 0. We found that the sparseness of both identity and viewpoint representations (obtained by averaging single image responses across viewpoints and identities, respectively, before computing the Gini index) increases significantly from ML-MF to AM (one-way ANOVA: viewpoint, F (2,21) ϭ 199.4, p ϭ 2 ϫ 10 Ϫ14 ; identity, F (2,72) ϭ 3477.39, p ϭ 2 ϫ 10 Ϫ72 ; Fig.…”

“…Sparseness. We computed the Gini index (Hurley and Rickard, 2009) on the basis of the normalized average responses to each identity (and to each viewpoint) in the face views set for all neurons in a given patch. The normalized responses represent how strongly each neuron responds to each identity (or viewpoint) as a fraction of their maximal response (in the image set); the response is sparse if a given stimulus evokes a maximal response in only a few neurons.…”

Depth Estimation From Stereo Image Based on Texture and Sparsity

“…By this a particular increase or decrease in sparsity is ensured with this measurements. The most commonly used and studied sparsity measures are the no rmlike measures [9] The measure simp ly calculates the number of non-zero coefficients…”

A Distributed Compressed Sensing-based Algorithm for the Joint Recovery of Signal Ensemble

“…The authors [4] examine and compare quantitatively several sparsity measures, and their findings show that the Gini index (GI) is the only measure that has all the defined properties. Hence, for the GI; given a vector x = [x 1 , ..., x N ], with its elements re-ordered from smallest to largest,…”

“…Although many natural signals of interest (i.e., smooth and piecewise smooth 1-D and 2-D signals with bounded variations) are not exactly sparse, norm |.| 0 may not be the desired measure of sparsity. Therefore, we derive an appropriate sparsity measure which utilizes the efficient GINI index (GI) introduced in [4].…”