Controlling Linguistic Style Aspects in Neural Language Generation

Ficler, Jessica; Goldberg, Yoav

doi:10.18653/v1/w17-4912

Cited by 239 publications

(222 citation statements)

References 25 publications

Supporting

Mentioning

222

Contrasting

Order By: Relevance

“…Content coupledness means how much style is tightly or loosely coupled to the content of original text. Ficler and Goldberg (2017) controlled different styles (e.g., descriptive, professional) in text variation, regardless of its coupledness to the semantics of text. However, it is often observed that content words are tightly coupled with its styles (Kang et al, 2019;Preoţiuc-Pietro and Ungar, 2018).…”

Section: Categorization Of Stylesmentioning

confidence: 99%

“…More severe semantic drift. The biggest challenge in collecting cross-style dataset (Kang et al, 2019) or controlling multiple styles in generation (Ficler and Goldberg, 2017) is to diversify style of text but at the same time preserve the meaning, in order to avoid semantic drift. It can be addressed by collecting text in parallel or preserving the meaning using various techniques.…”

Section: Challengesmentioning

confidence: 99%

See 1 more Smart Citation

GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding

Wang

Singh

Michael

et al. 2018

Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP

3,012

2,755

View full text Add to dashboard Cite

Every natural text is written in some style. The style is formed by a complex combination of different stylistic factors, including formality markers, emotions, metaphor, etc. Some factors implicitly reflect the author's personality, while others are explicitly controlled by the author's choices in order to achieve some personal or social goal. One cannot form a complete understanding of a text and its author without considering these factors. The factors combine and co-vary in complex ways to form styles. Studying the nature of the covarying combinations sheds light on stylistic language in general, sometimes called crossstyle language understanding. This paper provides a benchmark corpus (xSLUE) with an online platform (http://xslue.com) for crossstyle language understanding and evaluation. The benchmark contains text in 15 different styles and 23 classification tasks. For each task, we provide the fine-tuned classifier for further analysis. Our analysis shows that some styles are highly dependent on each other (e.g., impoliteness and offense), and some domains (e.g., tweets, political debates) are stylistically more diverse than others (e.g., academic manuscripts). We discuss the technical challenges of cross-style understanding and potential directions for future research: crossstyle modeling which shares the internal representation for low-resource or low-performance styles and other applications such as crossstyle generation.

show abstract

Section: Categorization Of Stylesmentioning

confidence: 99%

Section: Challengesmentioning

confidence: 99%

GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding

Wang

Singh

Michael

et al. 2018

Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP

3,012

2,755

View full text Add to dashboard Cite

show abstract

“…In each of these cases, the diversity in outputs is quite small given the constraints of the meaning representation, style is often constrained to interjections (like "yeah"), and there is no original style from which to transfer. Ficler and Goldberg (2017) investigate using stylistic parameters and content parameters to control text generation using a movie review dataset. Their stylistic parameters are created using wordlevel heuristics and they are successful in controlling these parameters in the outputs.…”

Section: Controlling Linguistic Featuresmentioning

confidence: 99%

Low Level Linguistic Controls for Style Transfer and Content Preservation

Gero

Kedzie

Reeve

et al. 2019

Proceedings of the 12th International Conference on Natural Language Generation

View full text Add to dashboard Cite

Despite the success of style transfer in image processing, it has seen limited progress in natural language generation. Part of the problem is that content is not as easily decoupled from style in the text domain. Curiously, in the field of stylometry, content does not figure prominently in practical methods of discriminating stylistic elements, such as authorship and genre. Rather, syntax and function words are the most salient features. Drawing on this work, we model style as a suite of low-level linguistic controls, such as frequency of pronouns, prepositions, and subordinate clause constructions. We train a neural encoder-decoder model to reconstruct reference sentences given only content words and the setting of the controls. We perform style transfer by keeping the content words fixed while adjusting the controls to be indicative of another style. In experiments, we show that the model reliably responds to the linguistic controls and perform both automatic and manual evaluations on style transfer. We find we can fool a style classifier 84% of the time, and that our model produces highly diverse and stylistically distinctive outputs. This work introduces a formal, extendable model of style that can add control to any neural text generation system.

show abstract

“…We consider the problem of automatic story continuation generation, i.e., how to generate story continuations conditioned on the story context. Inspired by recent work in controllable generation (Hu et al, 2017;Ficler and Goldberg, 2017), we propose a simple and effective modeling framework for controlled generation of multiple, diverse outputs based on interpretable control variables. Each control variable corresponds to an attribute of a sentence.…”

Section: Introductionmentioning

confidence: 99%

Generating Diverse Story Continuations with Controllable Semantics

Ding

Yu³

et al. 2019

Proceedings of the 3rd Workshop on Neural Generation and Translation

View full text Add to dashboard Cite

We propose a simple and effective modeling framework for controlled generation of multiple, diverse outputs. We focus on the setting of generating the next sentence of a story given its context. As controllable dimensions, we consider several sentence attributes, including sentiment, length, predicates, frames, and automatically-induced clusters. Our empirical results demonstrate: (1) our framework is accurate in terms of generating outputs that match the target control values;(2) our model yields increased maximum metric scores compared to standard n-best list generation via beam search; (3) controlling generation with semantic frames leads to a stronger combination of diversity and quality than other control variables as measured by automatic metrics. We also conduct a human evaluation to assess the utility of providing multiple suggestions for creative writing, demonstrating promising results for the potential of controllable, diverse generation in a collaborative writing system.

show abstract

Controlling Linguistic Style Aspects in Neural Language Generation

Cited by 239 publications

References 25 publications

GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding

GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding

Low Level Linguistic Controls for Style Transfer and Content Preservation

Generating Diverse Story Continuations with Controllable Semantics

Contact Info

Product

Resources

About