Exponential family Fisher vector for image classification

Abstract

a b s t r a c tOne of the fundamental problems in image classification is to devise models that allow us to relate the images to higher-level semantic concepts in an efficient and reliable way. A widely used approach consists on extracting local descriptors from the images and to summarize them into an image-level representation. Within this framework, the Fisher vector (FV) is one of the most robust signatures to date. In the FV, local descriptors are modeled as samples drawn from a mixture of Gaussian pdfs. An image is represented by a gradient vector characterizing the distributions of samples w.r.t. the model. Equipped with robust features like SIFT, the FV has shown state-of-the-art performance on different recognition problems. However, it is not clear how it should be applied when the feature space is clearly non-Euclidean, leading to heuristics that ignore the underlying structure of the space. In this paper we generalize the Gaussian FV to a broader family of distributions known as the exponential family. The model, termed exponential family Fisher vectors (eFV), provides a unified framework from which rich and powerful representations can be derived. Experimental results show the generality and flexibility of our approach.

Encoding the local connectivity patterns of fMRI for cognitive task and state classification

“…FV encoding is considered as an extension of BoW such that, rather than encoding the relative frequency of the descriptors, it encodes the information on distribution of the descriptors (Perronnin et al, 2010). Fisher kernels have been used to compute FVs utilizing a mechanism that incorporates generative probability models into discriminative classifiers in applications, including classification of protein domains (Jaakkola et al, 1999), action and event recognition (Oneata et al, 2013;Sekma et al, 2015), image classification (Liu et al, 2014;Simonyan et al, 2013;Sánchez and Redolfi, 2015) and 3D object retrieval (Savelonas et al, 2016). FVs have been applied to model effective connectivity of the networks using MRI and PET data (Zhou et al, 2016).…”

Hierarchical Gaussian Descriptors with Application to Person Re-Identification

“…Faraki et al proposed BOW, FV, and VLAD codings on covariance matrices [32], [33]. Sánchez et al extended Gaussian FV coding to a broader family of distribution, including Wishart distribution, which is a distribution of covariance matrices [34]. Ilea et al proposed a Riemannian FV coding based on the mixture model of Riemannian Gaussian distributions [35].…”

Aggregating binary local descriptors for image retrieval

“…An extension of the FV by using the BMM has been also carried in [73,64]. Our approach differs from the one proposed in [73] in the approximation of the square root of the inverse of the FIM (i.e., L λ ) .…”

“…the natural parameters of the Gaussian distribution rather than the mean and the variance parameters which are typically used in literature for the FV representation [56,54,63]. Unfortunately, the authors of [64] didn't experimentally compare the FVs obtained using or not the natural parameters.…”

“…In fact, in such case, s cos (x 1 , x 2 ) = 1 − 1 2 2 (x 1 , x 2 ) 2 , which implies that the ranked list of the results to a query is the same (i.e., 2 formulation. Furthermore, we have not experimentally evaluated the FVs computed using the gradient with respect to the natural parameters of a BMM or a GMM as described in [64], because the evaluation of the retrieval performance obtained using or not the natural parameters in the derivation of the score function is a more general topic which reserve to be further investigated outside the specific context of the encodings binary local features. Among the various baseline aggregation methods (i.e.…”