An Out-of-the-box Full-network Embedding for Convolutional Neural Networks

Garcia-Gasulla, Darío; Vilalta, Armand; Parés, Ferran; Moreno, J. L.; Ayguadé, Eduard; Labarta, Jesús; Cortés, Ulises; Suzumura, Toyotaro

doi:10.48550/arxiv.1705.07706

Cited by 3 publications

(5 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using a single layer embedding for that purpose would result in a rather poor topology, as fully-connected layers represent but a portion of all patterns learnt by the DNN, and only a small subset of neurons activate for each data instance. To guarantee that the graph representation contains a topology rich enough as to empower network analysis algorithms we use the full-network embedding [7], which produces a vector embedding including all convolutional and fully-connected layers of a CNN. This results in a much larger embedding space (composed by tens of thousands of dimensions), allowing us to generate larger and richer graphs.…”

Section: Graph Representation Of Vector Embeddingsmentioning

confidence: 99%

“…This process reduces noise and regularizes the embedding space. In the FNE, this discretization is done with a pair of constant thresholds (−0.25, 0.15) which determine if a feature is relevant by presence (1 implies an abnormally high activation) or is relevant by absence (-1 implies an abnormally low activation) for a given input data instance [7]. In our experiments we set more demanding thresholds (−2.0, 2.0), to make sure that the degree of sparsity of the graph is appropriate for network analysis methods.…”

Section: Full-network Embeddingmentioning

confidence: 99%

“…Extracted DNN representations are typically implemented through vectors, where the length of the vector equals to the number of neural features being used. These vector embedding spaces have been used to feed classifiers based on the instance-attribute paradigm (e.g., Support Vector Machines) [1,7]. Instead, in this paper we propose a graph based representation of those same embeddings spaces, with the goal of running a different family of algorithms; those based on the instance-instance paradigm, such as community detection algorithms.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Building Graph Representations of Deep Vector Embeddings

Garcia-Gasulla¹,

Vilalta²,

Parés³

et al. 2017

Preprint

Self Cite

View full text Add to dashboard Cite

Patterns stored within pre-trained deep neural networks compose large and powerful descriptive languages that can be used for many different purposes. Typically, deep network representations are implemented within vector embedding spaces, which enables the use of traditional machine learning algorithms on top of them. In this short paper we propose the construction of a graph embedding space instead, introducing a methodology to transform the knowledge coded within a deep convolutional network into a topological space (i.e., a network). We outline how such graph can hold data instances, data features, relations between instances and features, and relations among features. Finally, we introduce some preliminary experiments to illustrate how the resultant graph embedding space can be exploited through graph analytics algorithms.

show abstract

Section: Graph Representation Of Vector Embeddingsmentioning

confidence: 99%

Section: Full-network Embeddingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Building Graph Representations of Deep Vector Embeddings

Garcia-Gasulla¹,

Vilalta²,

Parés³

et al. 2017

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The optimal values of these thresholds can be found empirically for a labeled dataset [21]. Instead, we use threshold values shown to perform consistently across several domains [9].…”

Section: Full-network Embeddingmentioning

confidence: 99%

“…In fact, most approaches simply use a one-layer CNN embedding [7,8]. In this paper we explore the impact of using a Full-Network embedding (FNE) [9] to generate the required image embedding, replacing the one-layer embedding. We do so by integrating the FNE into the multimodal embedding pipeline defined by Kiros et al [1], which is based in the use of a Gated Recurrent Units neural network (GRU) [10] for text encoding and CNN for image encoding.…”

Section: Introductionmentioning

confidence: 99%

Full-Network Embedding in a Multimodal Embedding Pipeline

Vilalta,

Garcia-Gasulla,

Parés

et al. 2017

Preprint

Self Cite

View full text Add to dashboard Cite

The current state-of-the-art for image annotation and image retrieval tasks is obtained through deep neural networks, which combine an image representation and a text representation into a shared embedding space. In this paper we evaluate the impact of using the Full-Network embedding in this setting, replacing the original image representation in a competitive multimodal embedding generation scheme. Unlike the one-layer image embeddings typically used by most approaches, the Full-Network embedding provides a multi-scale representation of images, which results in richer characterizations. To measure the influence of the Full-Network embedding, we evaluate its performance on three different datasets, and compare the results with the original multimodal embedding generation scheme when using a one-layer image embedding, and with the rest of the state-of-the-art. Results for image annotation and image retrieval tasks indicate that the Full-Network embedding is consistently superior to the one-layer embedding. These results motivate the integration of the Full-Network embedding on any multimodal embedding generation scheme, something feasible thanks to the flexibility of the approach.

show abstract