A Modest Pareto Optimisation Analysis of Dependency Parsers in 2021

Anderson, Mark; Gómez-Rodríguez, Carlos

doi:10.18653/v1/2021.iwpt-1.12

Cited by 3 publications

(3 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our incremental-decoder-only models with LLMs as encoders are competitive against the BiLSTM-based version of the baseline (BiLSTM encoder, biaffine decoder), surpassing it on 7 out of 12 languages. However, they are a few points behind with respect to a version of the biaffine parser using RoBERTa encodings (which can be taken as a state-of-the-art system), consistent with existing comparisons of sequence-labeling parsers and biaffine parsers (Anderson and Gómez-Rodríguez, 2021). Put together, this seems to suggest that the challenge of incrementality falls mostly on the encoding side.…”

Section: Methodssupporting

confidence: 77%

On the Challenges of Fully Incremental Neural Dependency Parsing

Ezquerro,

Gómez-Rodríguez,

Vilares

2023

Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacifi

View full text Add to dashboard Cite

Since the popularization of BiLSTMs and Transformer-based bidirectional encoders, state-of-the-art syntactic parsers have lacked incrementality, requiring access to the whole sentence and deviating from human language processing. This paper explores whether fully incremental dependency parsing with modern architectures can be competitive. We build parsers combining strictly left-to-right neural encoders with fully incremental sequencelabeling and transition-based decoders. The results show that fully incremental parsing with modern architectures considerably lags behind bidirectional parsing, noting the challenges of psycholinguistically plausible parsing.

show abstract

Section: Methodssupporting

confidence: 77%

On the Challenges of Fully Incremental Neural Dependency Parsing

Ezquerro,

Gómez-Rodríguez,

Vilares

2023

Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacifi

View full text Add to dashboard Cite

show abstract

“…solving the quadratic problem: min w∈∆ K w T U w. To this end, using some well-known equalities in the RKHS 1 , we arrive at the following formula:…”

Section: Our Theoretical Developmentmentioning

confidence: 99%

“…On the other side, multi-objective optimization (MOO) [8] aims to optimize a set of objective functions and manifests itself in many real-world applications problems, such as in multi-task learning (MTL) [19,27], natural language processing [1], and reinforcement learning [10,25,24]. Leveraging the above insights, it is natural to ask: "Can we derive a probabilistic version of multi-objective • Demonstrate our algorithm is readily applicable in the context of multi-task learning.…”

Section: Introductionmentioning

confidence: 99%

Stochastic Multiple Target Sampling Gradient Descent

Phan¹,

Tran²,

Le³

et al. 2022

Preprint

View full text Add to dashboard Cite

Sampling from an unnormalized target distribution is an essential problem with many applications in probabilistic inference. Stein Variational Gradient Descent (SVGD) has been shown to be a powerful method that iteratively updates a set of particles to approximate the distribution of interest. Furthermore, when analysing its asymptotic properties, SVGD reduces exactly to a single-objective optimization problem and can be viewed as a probabilistic version of this single-objective optimization problem. A natural question then arises: "Can we derive a probabilistic version of the multi-objective optimization? ". To answer this question, we propose Stochastic Multiple Target Sampling Gradient Descent (MT-SGD), enabling us to sample from multiple unnormalized target distributions. Specifically, our MT-SGD conducts a flow of intermediate distributions gradually orienting to multiple target distributions, which allows the sampled particles to move to the joint high-likelihood region of the target distributions. Interestingly, the asymptotic analysis shows that our approach reduces exactly to the multiplegradient descent algorithm for multi-objective optimization, as expected. Finally, we conduct comprehensive experiments to demonstrate the merit of our approach to multi-task learning.

show abstract

Splitting EUD Graphs into Trees: A Quick and Clatty Approach

Anderson¹,

Gómez-Rodríguez²

2021

Proceedings of the 17th International Conference on Parsing Technologies and the IWPT 2021 Shared Task on Parsing Into Enhanced

Self Cite

View full text Add to dashboard Cite

We present the system submission from the FASTPARSE team for the EUD Shared Task at IWPT 2021. We engaged in the task last year by focusing on efficiency. This year we have focused on experimenting with new ideas on a limited time budget. Our system is based on splitting the EUD graph into several trees, based on linguistic criteria. We predict these trees using a sequence-labelling parser and combine them into an EUD graph. The results were relatively poor, although not a total disaster and could probably be improved with some polishing of the system's rough edges.

show abstract

A Modest Pareto Optimisation Analysis of Dependency Parsers in 2021

Cited by 3 publications

References 38 publications

On the Challenges of Fully Incremental Neural Dependency Parsing

On the Challenges of Fully Incremental Neural Dependency Parsing

Stochastic Multiple Target Sampling Gradient Descent

Splitting EUD Graphs into Trees: A Quick and Clatty Approach

Contact Info

Product

Resources

About