Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification

Shi, Yunsheng; Huang, Zhengjie; Feng, Shikun; Zhong, Hui; Wang, Wenjin; Sun, Yu

doi:10.24963/ijcai.2021/214

Cited by 246 publications

(147 citation statements)

References 0 publications

Supporting

Mentioning

107

Contrasting

Order By: Relevance

“…Recent works use Laplacian spectra, node degrees, and shortestpath lengths as positional encodings to expand attention to all nodes (Kreuzer et al, 2021;Dwivedi and Bresson, 2021;Rong et al, 2020;Ying et al, 2021). Several works also adapt attention mechanisms to GNNs (Yun et al, 2019;Cai and Lam, 2019;Hu et al, 2020;Baek et al, 2021;Veličković et al, 2018;Wang et al, 2021b;Zhang et al, 2020;Shi et al, 2021).…”

Section: Related Workmentioning

confidence: 99%

Rewiring with Positional Encodings for Graph Neural Networks

Brüel-Gabrielsson¹,

Yurochkin²,

Solomon³

2022

Preprint

View full text Add to dashboard Cite

Several recent works use positional encodings to extend the receptive fields of graph neural network (GNN) layers equipped with attention mechanisms. These techniques, however, extend receptive fields to the complete graph, at substantial computational cost and risking a change in the inductive biases of conventional GNNs, or require complex architecture adjustments. As a conservative alternative, we use positional encodings to expand receptive fields to any r-ring. Our method augments the input graph with additional nodes/edges and uses positional encodings as node and/or edge features. Thus, it is compatible with many existing GNN architectures. We also provide examples of positional encodings that are non-invasive, i.e., there is a oneto-one map between the original and the modified graphs. Our experiments demonstrate that extending receptive fields via positional encodings and a virtual fully-connected node significantly improves GNN performance and alleviates over-squashing using small r. We obtain improvements across models, showing state-ofthe-art performance even using older architectures than recent Transformer models adapted to graphs.

show abstract

Section: Related Workmentioning

confidence: 99%

Rewiring with Positional Encodings for Graph Neural Networks

Brüel-Gabrielsson¹,

Yurochkin²,

Solomon³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Experimental results of the Unified Message Passaging model (UniMP) [33] have shown that considering the label information on neighbors can bring significant improvements to prediction tasks. Therefore, inspired by the label propagation method of UniMP, we constructed a PSN to excavate extra information for the LOS prediction, with the assumption that similar patients would have similar LOS in the network.…”

Section: Patient Similarity Network Constructionmentioning

confidence: 99%

Network analytics and machine learning for predicting length of stay in elderly patients with chronic diseases at point of admission

Qiu

Wang

et al. 2022

BMC Med Inform Decis Mak

View full text Add to dashboard Cite

Background An aging population with a burden of chronic diseases puts increasing pressure on health care systems. Early prediction of the hospital length of stay (LOS) can be useful in optimizing the allocation of medical resources, and improving healthcare quality. However, the data available at the point of admission (PoA) are limited, making it difficult to forecast the LOS accurately. Methods In this study, we proposed a novel approach combining network analytics and machine learning to predict the LOS in elderly patients with chronic diseases at the PoA. Two networks, including multimorbidity network (MN) and patient similarity network (PSN), were constructed and novel network features were created. Five machine learning models (eXtreme Gradient Boosting, Gradient Boosting Decision Tree, Random Forest, Linear Support Vector Machine, and Deep Neural Network) with different input feature sets were developed to compare their performance. Results The experimental results indicated that the network features can bring significant improvements to the performances of the prediction models, suggesting that the MN and PSN are useful for LOS predictions. Conclusion Our predictive framework which integrates network science with data mining can forecast the LOS effectively at the PoA and provide decision support for hospital managers, which highlights the potential value of network-based machine learning in healthcare field.

show abstract

“…For an utterance, the action and intention of the speaker and interactions with other utterances in past, present, and future are crucial to model the context more precisely. Therefore, we construct a pSychological-Knowledge-Aware Interaction Graph (SKAIG) of utterances in a conversation, and then utilize the Graph Transformer (Shi et al, 2021) network to process SKAIG.…”

Section: Conversation-level Encodermentioning

confidence: 99%

“…In addition, we replace the original operation after the attention in Shi et al (2021) to a pointwise feed forward network proposed by Vaswani et al (2017). We denote the final output of the conversation as H L ∈ R N ×du .…”

Section: Graph Transformermentioning

confidence: 99%

“…These representations are produced by commonsense transformer COMET (Bosselut et al, 2019) which takes utterances and relations as inputs. As edges in SKAIG possess knowledge representations that require to be considered, we therefore utilize the graph transformer (Shi et al, 2021) for message passing. We then use the final outputs for classification.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Past, Present, and Future: Conversational Emotion Recognition through Structural Modeling of Psychological Knowledge

Li¹,

Lin²,

Fu³

et al. 2021

Findings of the Association for Computational Linguistics: EMNLP 2021

View full text Add to dashboard Cite

Conversational Emotion Recognition (CER) is a task to predict the emotion of an utterance in the context of a conversation. Although modeling the conversational context and interactions between speakers has been studied broadly, it is important to consider the speaker's psychological state, which controls the action and intention of the speaker. The state-of-the-art method introduces CommonSense Knowledge (CSK) to model psychological states in a sequential way (forwards and backwards). However, it ignores the structural psychological interactions between utterances. In this paper, we propose a pSychological-Knowledge-Aware Interaction Graph (SKAIG). In the locally connected graph, the targeted utterance will be enhanced with the information of action inferred from the past context and intention implied by the future context. The utterance is self-connected to consider the present effect from itself. Furthermore, we utilize CSK to enrich edges with knowledge representations and process the SKAIG with a graph transformer. Our method achieves state-of-theart and competitive performance on four popular CER datasets.

show abstract

Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification

Cited by 246 publications

References 0 publications

Rewiring with Positional Encodings for Graph Neural Networks

Rewiring with Positional Encodings for Graph Neural Networks

Network analytics and machine learning for predicting length of stay in elderly patients with chronic diseases at point of admission

Past, Present, and Future: Conversational Emotion Recognition through Structural Modeling of Psychological Knowledge

Contact Info

Product

Resources

About