Torch-Struct: Deep Structured Prediction Library

Rushton, Gérard

doi:10.18653/v1/2020.acl-demos.38

Cited by 50 publications

(46 citation statements)

References 25 publications

(22 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our second-order parsing algorithm has a theoretical time complexity of O(n 4 ), which is higher than the time complexity of O(n) of transition-based unsupervised parsers (Li et al, 2018) (Rush, 2020). In practice, our second-order parser runs very fast on GPU, requiring only several minutes to train.…”

Section: Limitationsmentioning

confidence: 99%

Second-Order Unsupervised Neural Dependency Parsing

Yang¹,

Jiang²,

Han³

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

View full text Add to dashboard Cite

Most of the unsupervised dependency parsers are based on first-order probabilistic generative models that only consider local parent-child information. Inspired by second-order supervised dependency parsing, we proposed a second-order extension of unsupervised neural dependency models that incorporate grandparent-child or sibling information. We also propose novel design of the neural parameterization and optimization methods of the dependency models. In secondorder models, the number of grammar rules grows cubically with the increase of vocabulary size, making it difficult to train lexicalized models that may contain thousands of words. To circumvent this problem while still benefiting from both second-order parsing and lexicalization, we use the agreement-based learning framework to jointly train a second-order unlexicalized model and a first-order lexicalized model. Experiments on multiple datasets show the effectiveness of our second-order models compared with recent state-of-the-art methods. Our joint model achieves a 10% improvement over the previous state-of-the-art parser on the full WSJ test set. 1

show abstract

Section: Limitationsmentioning

confidence: 99%

Second-Order Unsupervised Neural Dependency Parsing

Yang¹,

Jiang²,

Han³

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

View full text Add to dashboard Cite

show abstract

“…Typically, the CYK algorithm 3 can be directly used to solve this problem exactly: it first computes the score of the most likely parse; and then automatic differentiation is applied to recover the best tree structure t ⋆ (Eisner, 2016;Rush, 2020). This, however, relies on the original probability tensor T and is incompatible with our decomposed representation.…”

Section: Parsing With Td-pcfgsmentioning

confidence: 99%

PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbols

Yang¹,

Zhao²,

Tu³

2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

Probabilistic context-free grammars (PCFGs) with neural parameterization have been shown to be effective in unsupervised phrasestructure grammar induction. However, due to the cubic computational complexity of PCFG representation and parsing, previous approaches cannot scale up to a relatively large number of (nonterminal and preterminal) symbols. In this work, we present a new parameterization form of PCFGs based on tensor decomposition, which has at most quadratic computational complexity in the symbol number and therefore allows us to use a much larger number of symbols. We further use neural parameterization for the new form to improve unsupervised parsing performance. We evaluate our model across ten languages and empirically demonstrate the effectiveness of using more symbols.

show abstract

“…They introduced an additional module to determine which start and end tokens should be matched. We notice that the CRF implemented in PyTorch-Struct (Rush, 2020) has a different interface than usual CRF libraries in that it takes not two tensors for emission and transition scores, but rather one score tensor of the shape (batch size, sentence length, number of tags, number of tags). This allows one to incorporate even more prior knowledge in the structured prediction by setting a constraint mask as a function of not only a pair of tags, but also words on which the tags are assigned.…”

Section: Related Workmentioning

confidence: 99%

Masked Conditional Random Fields for Sequence Labeling

Wei¹,

Qi²,

He³

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

Conditional Random Field (CRF) based neural models are among the most performant methods for solving sequence labeling problems. Despite its great success, CRF has the shortcoming of occasionally generating illegal sequences of tags, e.g. sequences containing an "I-" tag immediately after an "O" tag, which is forbidden by the underlying BIO tagging scheme. In this work, we propose Masked Conditional Random Field (MCRF), an easy to implement variant of CRF that impose restrictions on candidate paths during both training and decoding phases. We show that the proposed method thoroughly resolves this issue and brings consistent improvement over existing CRF-based models with near zero additional cost.

show abstract

Torch-Struct: Deep Structured Prediction Library

Cited by 50 publications

References 25 publications

Second-Order Unsupervised Neural Dependency Parsing

Second-Order Unsupervised Neural Dependency Parsing

PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbols

Masked Conditional Random Fields for Sequence Labeling

Contact Info

Product

Resources

About