Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification

Shi, Yunsheng; Huang, Zhengjie; Wang, Wenjin; Zhong, Hui; Feng, Shikun; Sun, Yu

doi:10.48550/arxiv.2009.03509

Cited by 81 publications

(109 citation statements)

References 19 publications

Supporting

Mentioning

101

Contrasting

Order By: Relevance

“…However, the aforementioned models are required to load all the graph data into the memory at the same time, making training larger relation networks impossible. Gilmer et al (2017) Shi et al (2020) further show that using a Transformer-like operator to aggregate node features and the neighbor nodes' features gives a better performance than a simple average such as GraphSAGE or an attention mechanism such as Graph Attention Network (Veličković et al, 2017). To close the gap in the researches, we propose GTN-VF, which adopts the state-of-the-art Graph Neural Networks to model the relationships.…”

Section: Graph Neural Networkmentioning

confidence: 99%

Multivariate Realized Volatility Forecasting with Graph Neural Network

Chen¹,

Robert²

2021

Preprint

View full text Add to dashboard Cite

The existing publications demonstrate that the limit order book data is useful in predicting short-term volatility in stock markets. Since stocks are not independent, changes on one stock can also impact other related stocks. In this paper, we are interested in forecasting short-term realized volatility in a multivariate approach based on limit order book data and relational data. To achieve this goal, we introduce Graph Transformer Network for Volatility Forecasting. The model allows to combine limit order book features and an unlimited number of temporal and cross-sectional relations from different sources. Through experiments based on about 500 stocks from S&P 500 index, we find a better performance for our model than for other benchmarks.

show abstract

Section: Graph Neural Networkmentioning

confidence: 99%

Multivariate Realized Volatility Forecasting with Graph Neural Network

Chen¹,

Robert²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…At the second stage we construct fully-connected graph from all polyline representations d and apply graph transformer convolution [18]:…”

Section: Multimodal Modelmentioning

confidence: 99%

Estimating Uncertainty For Vehicle Motion Prediction on Yandex Shifts Dataset

Pustynnikov¹,

Eremeev²

2021

Preprint

View full text Add to dashboard Cite

Motion prediction of surrounding agents is an important task in context of autonomous driving since it is closely related to driver's safety. Vehicle Motion Prediction (VMP) track of Shifts Challenge ** focuses on developing models which are robust to distributional shift and able to measure uncertainty of their predictions. In this work we present the approach that significantly improved provided benchmark and took 2nd place on the leaderboard.

show abstract

“…After encoding the geometric tensors into SO(3)-invariant scalars t ij , we first embed them alone with other pre-defined node/edge attributes (h j , e ij ) into high-dimensional representations, and leverage an attention-based Graph Transformer architecture (Shi et al, 2020) to learn the SO(3)-invariant edgewise message embeddings m ij by propagating and aggregating information on the graph G X . The attention mechanism is introduced due to its powerful capacity in modeling those graphs with unknown topology.…”

Section: Graph Transformer Blockmentioning

confidence: 99%

SE(3) Equivariant Graph Neural Networks with Complete Local Frames

Wei¹,

Zhang²,

Du³

et al. 2021

Preprint

View full text Add to dashboard Cite

Modeling many-body systems has been a long-standing challenge in science, from classical and quantum physics to computational biology. Equivariance is a critical physical symmetry for many-body dynamic systems, which enables robust and accurate prediction under arbitrary reference transformations. In light of this, great efforts have been put on encoding this symmetry into deep neural networks, which significantly boosts the prediction performance of down-streaming tasks. Some general equivariant models which are computationally efficient have been proposed, however, these models have no guarantee on the approximation power and may have information loss. In this paper, we leverage insights from the scalarization technique in differential geometry to model many-body systems by learning the gradient vector fields, which are SE(3) and permutation equivariant. Specifically, we propose the Equivariant Vector Field Network (EVFN), which is built on a novel tuple of equivariant basis and the associated scalarization and vectorization layers. Since our tuple equivariant basis forms a complete basis, learning the dynamics with our EVFN has no information loss and no tensor operations are involved before the final vectorization, which reduces the complex optimization on tensors to a minimum. We evaluate our method on predicting trajectories of simulated Newton mechanics systems with both full and partially observed data, as well as the equilibrium state of small molecules (molecular conformation) evolving as a statistical mechanics system. Experimental results across multiple tasks demonstrate that our model achieves best or competitive performance on baseline models in various types of datasets.

show abstract

Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification

Cited by 81 publications

References 19 publications

Multivariate Realized Volatility Forecasting with Graph Neural Network

Multivariate Realized Volatility Forecasting with Graph Neural Network

Estimating Uncertainty For Vehicle Motion Prediction on Yandex Shifts Dataset

SE(3) Equivariant Graph Neural Networks with Complete Local Frames

Contact Info

Product

Resources

About