Multigraph Transformer for Free-Hand Sketch Recognition

Xu, Peng; Joshi, Chaitanya K.; Bresson, Xavier

doi:10.1109/tnnls.2021.3069230

Cited by 46 publications

(46 citation statements)

References 47 publications

(36 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Mathematical Theory Related to Attention Self-attention [44] is essentially processing each input as a fully-connected graph [51]. Therefore, as aforementioned, we start from a more general perspective of topological spaces [13] to rethink MAE, by regarding each image as a graph connecting patches instead of 2D pixel grid.…”

Section: Related Workmentioning

confidence: 99%

How to Understand Masked Autoencoders

Cao¹,

Xu²,

Clifton³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Masked Autoencoders (MAE) Are Scalable Vision Learners" revolutionizes the self-supervised learning method in that it not only achieves the state-of-the-art for image pre-training, but is also a milestone that bridges the gap between visual and linguistic masked autoencoding (BERT-style) pre-trainings. However, to our knowledge, to date there are no theoretical perspectives to explain the powerful expressivity of MAE. In this paper, we, for the first time, propose a unified theoretical framework that provides a mathematical understanding for MAE. Specifically, we explain the patch-based attention approaches of MAE using an integral kernel under a non-overlapping domain decomposition setting. To help the research community to further comprehend the main reasons of the great success of MAE, based on our framework, we pose five questions and answer them with mathematical rigor using insights from operator theory.

show abstract

Section: Related Workmentioning

confidence: 99%

How to Understand Masked Autoencoders

Cao¹,

Xu²,

Clifton³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…We propose an approach to integrate a concise encoding of knowledge graphs into a Transformer-based decoder architecture for knowledge-grounded dialogue generation. Transformers for natural language generation can be viewed as graph neural networks which use selfattention (Veličković et al, 2018) for neighborhood aggregation on fully-connected word graphs (Xu et al, 2019). We utilize this relationship and restrict the self-attention weights to match the underlying graph structure.…”

Section: Contributionsmentioning

confidence: 99%

Space Efficient Context Encoding for Non-Task-Oriented Dialogue Generation with Graph Attention Transformer

Galetzka¹,

Rose²,

Schlangen³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

To improve the coherence and knowledge retrieval capabilities of non-task-oriented dialogue systems, recent Transformer-based models aim to integrate fixed background context. This often comes in the form of knowledge graphs, and the integration is done by creating pseudo utterances through paraphrasing knowledge triples, added into the accumulated dialogue context. However, the context length is fixed in these architectures, which restricts how much background or dialogue context can be kept. In this work, we propose a more concise encoding for background context structured in the form of knowledge graphs, by expressing the graph connections through restrictions on the attention weights. The results of our human evaluation show that this encoding reduces space requirements without negative effects on the precision of reproduction of knowledge and perceived consistency. Further, models trained with our proposed context encoding generate dialogues that are judged to be more comprehensive and interesting.

show abstract

“…The main advantage of the transformer is its global computation and perfect memory mechanism, which makes it more suitable than Recurrent Neural Networks on long sequences. Nowadays, transformer is widely studied in various tasks, including natural language processing [30], [31], computer vision [32], [33] and data mining [34]. Inspired by the success of these works, we apply a transformer to learn the long-rang spatial distribution of fine-grained traffic flow.…”

Section: Transformer Architecturementioning

confidence: 99%

Road Network Guided Fine-Grained Urban Traffic Flow Inference

Liu¹,

Liu²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

Accurate inference of fine-grained traffic flow from coarse-grained one is an emerging yet crucial problem, which can help greatly reduce the number of traffic monitoring sensors for cost savings. In this work, we notice that traffic flow has a high correlation with road network, which was either completely ignored or simply treated as an external factor in previous works. To facilitate this problem, we propose a novel Road-Aware Traffic Flow Magnifier (RATFM) that explicitly exploits the prior knowledge of road networks to fully learn the roadaware spatial distribution of fine-grained traffic flow. Specifically, a multi-directional 1D convolutional layer is first introduced to extract the semantic feature of the road network. Subsequently, we incorporate the road network feature and coarse-grained flow feature to regularize the short-range spatial distribution modeling of road-relative traffic flow. Furthermore, we take the road network feature as a query to capture the long-range spatial distribution of traffic flow with a transformer architecture. Benefiting from the road-aware inference mechanism, our method can generate high-quality fine-grained traffic flow maps. Extensive experiments on three real-world datasets show that the proposed RATFM outperforms state-of-the-art models under various scenarios. Our source code and datasets would be released after peer review.

show abstract

Multigraph Transformer for Free-Hand Sketch Recognition

Cited by 46 publications

References 47 publications

How to Understand Masked Autoencoders

How to Understand Masked Autoencoders

Space Efficient Context Encoding for Non-Task-Oriented Dialogue Generation with Graph Attention Transformer

Road Network Guided Fine-Grained Urban Traffic Flow Inference

Contact Info

Product

Resources

About