Median Activation Functions for Graph Neural Networks

Ruiz, Luana; Gama, Fernando; Marqués, Antonio G.; Ribeiro, Alejandro

doi:10.1109/icassp.2019.8683472

Cited by 7 publications

(6 citation statements)

References 21 publications

(17 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

Section: Introductionmentioning

confidence: 99%

“…Regarding This work in this paper was supported by NSF CCF 1717120, ARO W911NF1710438, ARL DCIST CRA W911NF-17-2-0181, ISTC-WAS and Intel DevCloud, and Spanish MINECO grant TEC2016-75361-R. Preliminary results have been submitted for publication at the ICASSP19 conference [1]. L. Ruiz, F. Gama activation functions, GNNs utilize the same functions as CNNs -rectified linear units (ReLUs), sigmoids, and hyperbolic tangents-, and, like in CNNs, these functions are either applied locally [4], [5] or within local neighborhoods when mixed with nonlinear pooling operations [2], [3], [7].…”

mentioning

confidence: 99%

See 1 more Smart Citation

Invariance-Preserving Localized Activation Functions for Graph Neural Networks

Ruiz

Gama

Marqués

et al. 2020

IEEE Trans. Signal Process.

Self Cite

View full text Add to dashboard Cite

Graph signals are signals with an irregular structure that can be described by a graph. Graph neural networks (GNNs) are information processing architectures tailored to these graph signals and made of stacked layers that compose graph convolutional filters with nonlinear activation functions. Graph convolutions endow GNNs with invariance to permutations of the graph nodes' labels. In this paper, we consider the design of trainable nonlinear activation functions that take into consideration the structure of the graph. This is accomplished by using graph median filters and graph max filters, which mimic linear graph convolutions and are shown to retain the permutation invariance of GNNs. We also discuss modifications to the backpropagation algorithm necessary to train local activation functions. The advantages of localized activation function architectures are demonstrated in four numerical experiments: source localization on synthetic graphs, authorship attribution of 19th century novels, movie recommender systems and scientific article classification. In all cases, localized activation functions are shown to improve model capacity.

show abstract

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

Invariance-Preserving Localized Activation Functions for Graph Neural Networks

Ruiz

Gama

Marqués

et al. 2020

IEEE Trans. Signal Process.

Self Cite

View full text Add to dashboard Cite

show abstract

“…On the other hand, CNNs can only process information defined on Euclidean spaces, which makes them unsuitable for handling data on irregular domains, i.e., where the convolution operation is not trivially defined. Graph neural networks (GNNs) emerged as tools to generalize CNNs to non-Euclidean domains by leveraging consolidated developments of graph signal processing and exploiting the shift operator to perform graph convolutions [3], [4].…”

Section: Introductionmentioning

confidence: 99%

Graphon Pooling in Graph Neural Networks

Parada-Mayorga

Ruiz

Ribeiro

2021

2020 28th European Signal Processing Conference (EUSIPCO)

Self Cite

View full text Add to dashboard Cite

Graph neural networks (GNNs) have been used effectively in different applications involving the processing of signals on irregular structures modeled by graphs. Relying on the use of shift-invariant graph filters, GNNs extend the operation of convolution to graphs. However, the operations of pooling and sampling are still not clearly defined and the approaches proposed in the literature either modify the graph structure in a way that does not preserve its spectral properties, or require defining a policy for selecting which nodes to keep. In this work, we propose a new strategy for pooling and sampling on GNNs using graphons which preserves the spectral properties of the graph. To do so, we consider the graph layers in a GNN as elements of a sequence of graphs that converge to a graphon. In this way we have no ambiguity in the node labeling when mapping signals from one layer to the other and a spectral representation that is consistent throughout the layers. We evaluate this strategy in a synthetic and a real-world numerical experiment where we show that graphon pooling GNNs are less prone to overfitting and improve upon other pooling techniques, especially when the dimensionality reduction ratios between layers is large.

show abstract

“…Both of these problems can be overcome if we use a Graph Neural Networks [10,11,12,13,14]. In particular, aggregation graph neural networks (aggregation GNNs) [13] are especially suited to teams of agents operating over a physical network because the architecture operates in an entirely local fashion involving only communication between nearby neighbors.…”

Section: Introductionmentioning

confidence: 99%

Learning Decentralized Controllers for Robot Swarms with Graph Neural Networks

Tolstaya,

Gama,

Paulos

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

We consider the problem of finding distributed controllers for large networks of mobile robots with interacting dynamics and sparsely available communications. Our approach is to learn local controllers that require only local information and communications at test time by imitating the policy of centralized controllers using global information at training time. By extending aggregation graph neural networks to time varying signals and time varying network support, we learn a single common local controller which exploits information from distant teammates using only local communication interchanges. We apply this approach to the problem of flocking to demonstrate performance on communication graphs that change as the robots move. We examine how a decreasing communication radius and faster velocities increase the value of multi-hop information.

show abstract

Median Activation Functions for Graph Neural Networks

Cited by 7 publications

References 21 publications

Invariance-Preserving Localized Activation Functions for Graph Neural Networks

Invariance-Preserving Localized Activation Functions for Graph Neural Networks

Graphon Pooling in Graph Neural Networks

Learning Decentralized Controllers for Robot Swarms with Graph Neural Networks

Contact Info

Product

Resources

About