Improving the Naturalness and Expressivity of Language Generation for Spanish

Barros, Cristina; Gkatzia, Dimitra; Lloret, Elena

doi:10.18653/v1/w17-3505

Cited by 5 publications

(4 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Section: Resultsmentioning

confidence: 99%

“…For this experiment, our system achieves 100% accuracy when evaluated on the additional test set. Furthermore, our model contributes to the improvement of naturalness and expressivity of NLG (Barros et al, 2017).Base form-Inflected form contar-cuenta; errar-yerra; haber-he; hacer-haz; olerhuele; ir-ve; oír-oye; decir-di Table 4: Variability of inflection in the imperative mood for the 2nd person singular of the present.Error Analysis: Although our system obtains almost 100% accuracy, it fails on the inflection of the participles of extremely rare irregular verbs (e.g., verb: ejabrir → generated: ejabrido → correct: ejabierto). These errors could be corrected by adding specific rules for these cases.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Proceedings of the First Workshop on Subword and Character Level Models in NLP

2017

View full text Add to dashboard Cite

IntroductionTraditional NLP starts with a hand-engineered layer of representation, the level of tokens or words. A tokenization component first breaks up the text into units using manually designed rules. Tokens are then processed by components such as word segmentation, morphological analysis and multiword recognition. The heterogeneity of these components makes it hard to create integrated models of both structure within tokens (e.g., morphology) and structure across multiple tokens (e.g., multi-word expressions). This approach can perform poorly (i) for morphologically rich languages, (ii) for noisy text, (iii) for languages in which the recognition of words is difficult and (iv) for adaptation to new domains; and (v) it can impede the optimization of preprocessing in end-to-end learning.The workshop provides a forum for discussing recent advances as well as future directions on sub-word and character-level natural language processing and representation learning that address these problems.We received 37 submissions, out of which we accepted 24 as papers and 4 as extended abstracts. AbstractMost of neural language models use different kinds of embeddings for word prediction. While word embeddings can be associated to each word in the vocabulary or derived from characters as well as factored morphological decomposition, these word representations are mainly used to parametrize the input, i.e. the context of prediction. This work investigates the effect of using subword units (character and factored morphological decomposition) to build output representations for neural language modeling. We present a case study on Czech, a morphologically-rich language, experimenting with different input and output representations. When working with the full training vocabulary, despite unstable training, our experiments show that augmenting the output word representations with character-based embeddings can significantly improve the performance of the model. Moreover, reducing the size of the output look-up table, to let the character-based embeddings represent rare words, brings further improvement. IntroductionMost of neural language models, such as n-gram models (Bengio et al., 2003) are word based and rely on the definition of a finite vocabulary V. Therefore, a look-up table maps each wordw ∈ V to a vector of real features, and is stored in a matrix. While this approach yields significant improvement for a variety of tasks and languages, see for instance (Schwenk, 2007) in speech recognition and (Le et al., 2012; Devlin et al., 2014; in machine translation, it induces several limitations.For morphologically-rich languages, like Czech or German, the lexical coverage is still an important issue, since there is a combinatorial explosion of word forms, most of which are hardly observed on training data. On the one hand, growing the look-up table is not a solution, since it would increase the number of parameters without having enough training examples for a proper estimation. On the other hand, rare words can be replaced...

show abstract

Section: Resultsmentioning

confidence: 99%

mentioning

confidence: 99%

Proceedings of the First Workshop on Subword and Character Level Models in NLP

2017

View full text Add to dashboard Cite

show abstract

“…In recent years, the availability of large-scale vision and language corpora such as RefCOCO (Kazemzadeh et al, 2014 ) and more general advances in Computer Vision and neural language modeling have alleviated some of these problems, allowing to extend the REG task to more natural inputs like Figure 3 . In this visual REG paradigm, the goal is to generate descriptions using raw visual representations of objects in natural images (Mao et al, 2016 ; Yu et al, 2016 , 2017 ; Zarrieß and Schlangen, 2016 , 2018 , 2019 ; Liu et al, 2017 , 2020 ; Luo and Shakhnarovich, 2017 ; Li and Jiang, 2018 ; Tanaka et al, 2019 ; Kim et al, 2020 ; Panagiaris et al, 2020 , 2021 ; Schüz and Zarrieß, 2021 ; Sun et al, 2022 ).…”

Section: The Reg Taskmentioning

confidence: 99%

Rethinking symbolic and visual context in Referring Expression Generation

Schüz¹,

Gatt²,

Zarrieß³

2023

Front. Artif. Intell.

View full text Add to dashboard Cite

Situational context is crucial for linguistic reference to visible objects, since the same description can refer unambiguously to an object in one context but be ambiguous or misleading in others. This also applies to Referring Expression Generation (REG), where the production of identifying descriptions is always dependent on a given context. Research in REG has long represented visual domains through symbolic information about objects and their properties, to determine identifying sets of target features during content determination. In recent years, research in visual REG has turned to neural modeling and recasted the REG task as an inherently multimodal problem, looking at more natural settings such as generating descriptions for objects in photographs. Characterizing the precise ways in which context influences generation is challenging in both paradigms, as context is notoriously lacking precise definitions and categorization. In multimodal settings, however, these problems are further exacerbated by the increased complexity and low-level representation of perceptual inputs. The main goal of this article is to provide a systematic review of the types and functions of visual context across various approaches to REG so far and to argue for integrating and extending different perspectives on visual context that currently co-exist in research on REG. By analyzing the ways in which symbolic REG integrates context in rule-based approaches, we derive a set of categories of contextual integration, including the distinction between positive and negative semantic forces exerted by context during reference generation. Using this as a framework, we show that so far existing work in visual REG has considered only some of the ways in which visual context can facilitate end-to-end reference generation. Connecting with preceding research in related areas, as possible directions for future research, we highlight some additional ways in which contextual integration can be incorporated into REG and other multimodal generation tasks.

show abstract

“…A far messier criterion name is Coherence, some definitions referring to structure (underlined text below) and theme/topic (dotted underline), some just to one of the two, and others to neither (last three examples): "[whether] the poem [is] thematically structured" (Van de Cruys, 2020); "measures if a question is coherent with previous ones" (Chai and Wan, 2020); "measures ability of the dialogue system to produce responses consistent with the topic of conversation" (Santhanam and Shaikh, 2019); "measures how much the response is comprehensible and relevant to a user's request" (Yi et al, 2019); "refers to the meaning of the generated sentence, so that a sentence with no meaning would be rated with a 1 and a sentence with a full meaning would be rated with a 5" (Barros et al, 2017); "measures [a conversation's] grammaticality and fluency" (Juraska et al, 2019); "concerns coherence and readability" (Murray et al, 2010). The inverse is also common, where the same definition is used with different criterion names.…”

Section: Quality Criterion Namesmentioning

confidence: 99%

Disentangling the Properties of Human Evaluation Methods: A Classification System to Support Comparability, Meta-Evaluation and Reproducibility Testing

Belz,

Mille,

Howcroft

2020

Proceedings of the 13th International Conference on Natural Language Generation

View full text Add to dashboard Cite

Current standards for designing and reporting human evaluations in NLP mean it is generally unclear which evaluations are comparable and can be expected to yield similar results when applied to the same system outputs. This has serious implications for reproducibility testing and meta-evaluation, in particular given that human evaluation is considered the gold standard against which the trustworthiness of automatic metrics is gauged. Using examples from NLG, we propose a classification system for evaluations based on disentangling (i) what is being evaluated (which aspect of quality), and (ii) how it is evaluated in specific (a) evaluation modes and (b) experimental designs. We show that this approach provides a basis for determining comparability, hence for comparison of evaluations across papers, meta-evaluation experiments, reproducibility testing.

show abstract

Improving the Naturalness and Expressivity of Language Generation for Spanish

Cited by 5 publications

References 25 publications

Proceedings of the First Workshop on Subword and Character Level Models in NLP

Proceedings of the First Workshop on Subword and Character Level Models in NLP

Rethinking symbolic and visual context in Referring Expression Generation

Disentangling the Properties of Human Evaluation Methods: A Classification System to Support Comparability, Meta-Evaluation and Reproducibility Testing

Contact Info

Product

Resources

About