Unsupervised Controllable Text Formalization

Abstract

We propose a novel framework for controllable natural language transformation. Realizing that the requirement of parallel corpus is practically unsustainable for controllable generation tasks, an unsupervised training scheme is introduced. The crux of the framework is a deep neural encoder-decoder that is reinforced with text-transformation knowledge through auxiliary modules (called scorers). These scorers, based on off-the-shelf language processing tools, decide the learning scheme of the encoder-decoder based on its actions. We apply this framework for the text-transformation task of formalizing an input text by improving its readability grade; the degree of required formalization can be controlled by the user at run-time. Experiments on public datasets demonstrate the efficacy of our model towards: (a) transforming a given text to a more formal style, and (b) varying the amount of formalness in the output text based on the specified input control. Our code and datasets are released for academic use.

Reflection-based Word Attribute Transfer

“…The style transfer task presented in previous studies (Jain et al 2019;Logeswaran et al 2018) resembles ours. In style transfer, the text style of the input sentences is changed.…”

“…In style transfer, the text style of the input sentences is changed. For instance, Jain et al (2019) transferred the style from formal to informal sentences. Logeswaran et al (2018) transferred sentences by controlling attributes such as mood and tense.…”

Adapting Language Models for Non-Parallel Author-Stylized Rewriting

“…Stylized Text Generation: In recent times, several explorations that aim to generate stylized text define a psycholinguistic aspect, like, formality or sentiment (Shen et al 2017;Ficler and Goldberg 2017;Jain et al 2019) and transfer text along this dimension. The approaches themselves can range from completely supervised, which is contingent on the availability of parallel data (Ficler and Goldberg 2017), to unsupervised (Shen et al 2017;Li et al 2018;Jain et al 2019).…”

“…Stylized Text Generation: In recent times, several explorations that aim to generate stylized text define a psycholinguistic aspect, like, formality or sentiment (Shen et al 2017;Ficler and Goldberg 2017;Jain et al 2019) and transfer text along this dimension. The approaches themselves can range from completely supervised, which is contingent on the availability of parallel data (Ficler and Goldberg 2017), to unsupervised (Shen et al 2017;Li et al 2018;Jain et al 2019). Some of the influential unsupervised approaches include (a) using readily available classificationbased discriminators to guide the process of generation (Fu et al 2018), (b) using simple linguistic rules to achieve alignment with the target style (Li et al 2018), or (c) using auxiliary modules (called scorers) that score the gen-eration process on aspects like fluency, formality and semantic relatedness while deciding on the learning scheme of the encoder-decoder network (Jain et al 2019).…”

“…Recently, stylized text generation (Hovy 1990;Inkpen and Hirst 2006) and style transfer (Li et al 2018;Prabhumoye et al 2018;Fu et al 2018) have gained traction; both these tasks aim to generate realizations of an input text that align to a target style. A majority of the work here is focused around generating text with different levels of sentiment (Shen et al 2017;Ficler and Goldberg 2017) and formality (Jain et al 2019) Copyright c 2020, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.…”