Continuous Prompt Tuning for Russian: How to Learn Prompts Efficiently with RuGPT3?

Konodyuk, Nikita; Tikhonova, Maria

doi:10.1007/978-3-031-15168-2_3

Cited by 3 publications

(3 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This idea was further developed in (Liu et al, 2021), where the authors introduce the concept of deep prompt tuning, which involves adding prompts in different layers as prefix tokens. In (Konodyuk and Tikhonova, 2022), the authors studied the applicability of the prompt-tuning method for the Russian language: they showed that it could be a good alternative to model training techniques.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Parameter-Efficient Tuning of Transformer Models for Anglicism Detection and Substitution in Russian

Lukichev,

Kryanina,

Bystrova

et al. 2023

Computational Linguistics and Intellectual Technologies”

View full text Add to dashboard Cite

This article is devoted to the problem of Anglicisms in texts in Russian: the tasks of detection and automatic rewriting of the text with the substitution of Anglicisms by their Russian-language equivalents. Within the framework of the study, we present a parallel corpus of Anglicisms and models that identify Anglicisms in the text and replace them with the Russian equivalent, preserving the stylistics of the original text.

show abstract

Section: Related Workmentioning

confidence: 99%

“…As a result, we obtained a relatively modest but high-quality dataset. In addition, it should be noted that we took into account the current dataset size and selected suitable methods, such as prompt-tuning and LoRA (see section 5), which can be successfully applied to such amounts of data (Konodyuk and Tikhonova, 2022).…”

Section: Sentence (English)mentioning

confidence: 99%

Parameter-Efficient Tuning of Transformer Models for Anglicism Detection and Substitution in Russian

Lukichev,

Kryanina,

Bystrova

et al. 2023

Computational Linguistics and Intellectual Technologies”

View full text Add to dashboard Cite

show abstract

“…RuPrompts This baseline is based on the ruPrompts library 12 for fast language model tuning via automatic prompt search. The Continuous Prompt Tuning method (Konodyuk and Tikhonova, 2021) consists in training embeddings corresponding to the prompts. Such approach is cheaper than classic fine-tuning of big language models.…”

Section: Baselinesmentioning

confidence: 99%

RUSSE-2022: Findings of the First Russian Detoxification Shared Task Based on Parallel Corpora

Dementieva¹,

Logacheva²,

Nikishina³

et al. 2022

Computational Linguistics and Intellectual Technologies

View full text Add to dashboard Cite

Text detoxification is the task of rewriting a toxic text into a neutral text while preserving its original content. It has a wide range of applications, e.g. moderation of output of neural chatbots or suggesting less emotional version of posts on social networks. This paper provides a description of RUSSE-2022 competition of detoxification methods for the Russian language. This is the first competition which features (i) parallel training data and (ii) manual evaluation. We describe the setup of the competition, the solutions of the participating teams and analyse their performance. In addition to that, the large-scale evaluation allows us to analyse the performance of automatic evaluation metrics.

show abstract

Paraphrasers and Classifiers: Controllable Text Generation for Text Style Transfer

Orlov,

Apishev

2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Continuous Prompt Tuning for Russian: How to Learn Prompts Efficiently with RuGPT3?

Cited by 3 publications

References 9 publications

Parameter-Efficient Tuning of Transformer Models for Anglicism Detection and Substitution in Russian

Parameter-Efficient Tuning of Transformer Models for Anglicism Detection and Substitution in Russian

RUSSE-2022: Findings of the First Russian Detoxification Shared Task Based on Parallel Corpora

Paraphrasers and Classifiers: Controllable Text Generation for Text Style Transfer

Contact Info

Product

Resources

About