HeteGCN

Ragesh, Rahul; Sellamanickam, Sundararajan; Iyer, Arun; Bairi, Ramakrishna; Lingam, Vijay

doi:10.1145/3437963.3441746

Cited by 54 publications

(17 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…BoW-based and sequence-based models are inherently inductive. Ragesh et al [RSI+21] have evaluated a variant of TextGCN that is capable of inductive learning, which we also include in our results.…”

Section: Methodsmentioning

confidence: 99%

“…For example, TextGCN [YML19] first induces a synthetic word-document co-occurrence graph across the entire corpus and then applies a graph neural network (GNN) to perform the classification task. In addition to TextGCN, there are follow-up works that employ a similar strategy, such as HeteGCN [RSI+21], TensorGCN [LYZ+20], and HyperGAT [DWL+20], which we collectively call graph-based models.…”

Section: Wide Multilayer Perceptrons For Text Classificationmentioning

confidence: 99%

“…Modern approaches exploit this idea in combination with graph neural networks (GNN) [Ham20]. Examples of GNN-based methods operating on a word-document co-occurrence graph are TextGCN [YML19] and its successor TensorGCN [LYZ+20] aswell as HeteGCN [RSI+21], HyperGAT [DWL+20], and DADGNN [LGG+21]. We briefly discuss these models.…”

Section: Graph-based Modelsmentioning

confidence: 99%

“…This may be prohibitive in practical applications, as each batch of new documents would require retraining of the model. When these methods are adapted for inductive learning, where the test set is unseen, they achieve considerably lower scores [RSI+21].…”

Section: Graph-based Modelsmentioning

confidence: 99%

“…Follow-up approaches already relax this constraint and are able to deal with new documents being inserted after training. Those include HeteGCN [RSI+21], HyperGAT [DWL+20], and DADGNN [LGG+21], which will be described in more detail in Chapter 5.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Representation Learning for Texts and Graphs

Galke

View full text Add to dashboard Cite

[...] This thesis is situated between natural language processing and graph representation learning and investigates selected connections. First, we introduce matrix embeddings as an efficient text representation sensitive to word order. [...] Experiments with ten linguistic probing tasks, 11 supervised, and five unsupervised downstream tasks reveal that vector and matrix embeddings have complementary strengths and that a jointly trained hybrid model outperforms both. Second, a popular pretrained language model, BERT, is distilled into matrix embeddings. [...] The results on the GLUE benchmark show that these models are competitive with other recent contextualized language models while being more efficient in time and space. Third, we compare three model types for text classification: bag-of-words, sequence-, and graph-based models. Experiments on five datasets show that, surprisingly, a wide multilayer perceptron on top of a bag-of-words representation is competitive with recent graph-based approaches, questioning the necessity of graphs synthesized from the text. [...] Fourth, we investigate the connection between text and graph data in document-based recommender systems for citations and subject labels. Experiments on six datasets show that the title as side information improves the performance of autoencoder models. [...] We find that the meaning of item co-occurrence is crucial for the choice of input modalities and an appropriate model. Fifth, we introduce a generic framework for lifelong learning on evolving graphs in which new nodes, edges, and classes appear over time. [...] The results show that by reusing previous parameters in incremental training, it is possible to employ smaller history sizes with only a slight decrease in accuracy compared to training with complete history. Moreover, weighting the binary cross-entropy loss function is crucial to mitigate the problem of class imbalance when detecting newly emerging classes. [...]

show abstract

Section: Methodsmentioning

confidence: 99%