Evolution of Graph Classifiers

Domingue, Miguel; Dhamdhere, Rohan; Kanamarlapudi, Naga Durga Harish; Raghupathi, Sunand; Ptucha, Raymond

doi:10.1109/wnyipw.2019.8923110

Cited by 1 publication

(1 citation statement)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We compare ET with the current state-of-the-art approaches for the mentioned datasets, which include WKPI-kmeans [49], WKPI-kcenters [49], DSGCN [50], HGP-SL [51], U2GNN [52], and Evolution Graph Classifier (EvoG) [53]. Additionally, approaches that are close to the baselines are included to further contrast the performance of our model.…”

Section: F2 Experimental Evaluationmentioning

confidence: 99%

Energy Transformer

Hoover¹,

Liang²,

Pham³

et al. 2023

Preprint

View full text Add to dashboard Cite

Transformers have become the de facto models of choice in machine learning, typically leading to impressive performance on many applications. At the same time, the architectural development in the transformer world is mostly driven by empirical findings, and the theoretical understanding of their architectural building blocks is rather limited. In contrast, Dense Associative Memory models or Modern Hopfield Networks have a well-established theoretical foundation, but have not yet demonstrated truly impressive practical results. We propose a transformer architecture that replaces the sequence of feedforward transformer blocks with a single large Associative Memory model. Our novel architecture, called Energy Transformer (or ET for short), has many of the familiar architectural primitives that are often used in the current generation of transformers. However, it is not identical to the existing architectures. The sequence of transformer layers in ET is purposely designed to minimize a specifically engineered energy function, which is responsible for representing the relationships between the tokens. As a consequence of this computational principle, the attention in ET is different from the conventional attention mechanism. In this work, we introduce the theoretical foundations of ET, explore it's empirical capabilities using the image completion task, and obtain strong quantitative results on the graph anomaly detection task.

show abstract

Section: F2 Experimental Evaluationmentioning

confidence: 99%