Character Sequence Models for Colorful Words

Kawakami, Kazuya; Dyer, Chris; Routledge, Bryan R.; Smith, Noah A.

doi:10.18653/v1/d16-1202

Cited by 12 publications

(11 citation statements)

References 21 publications

(24 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The linguistic behavior of the players exhibits many of the intricacies of language in general, including not just the context dependence and cognitive complexity discussed above, but also compositionality, vagueness, and ambiguity. While many previous data sets feature descriptions of individual colors (Cook et al, 2005;Munroe, 2010;Kawakami et al, 2016), situating colors in a communicative context elicits greater variety in language use, including negations, comparatives, superlatives, metaphor, and shared associations.…”

Section: Introductionmentioning

confidence: 99%

Colors in Context: A Pragmatic Neural Model for Grounded Language Understanding

Monroe

Hawkins

Goodman

et al. 2017

TACL

100

134

View full text Add to dashboard Cite

We present a model of pragmatic referring expression interpretation in a grounded communication task (identifying colors from descriptions) that draws upon predictions from two recurrent neural network classifiers, a speaker and a listener, unified by a recursive pragmatic reasoning framework. Experiments show that this combined pragmatic model interprets color descriptions more accurately than the classifiers from which it is built, and that much of this improvement results from combining the speaker and listener perspectives. We observe that pragmatic reasoning helps primarily in the hardest cases: when the model must distinguish very similar colors, or when few utterances adequately express the target color. Our findings make use of a newly-collected corpus of human utterances in color reference games, which exhibit a variety of pragmatic behaviors. We also show that the embedded speaker model reproduces many of these pragmatic behaviors.

show abstract

Section: Introductionmentioning

confidence: 99%

Colors in Context: A Pragmatic Neural Model for Grounded Language Understanding

Monroe

Hawkins

Goodman

et al. 2017

TACL

100

134

View full text Add to dashboard Cite

show abstract

“…3 One natural extension is the use of character-level sequence modeling to capture complex morphology (e.g., "-ish" in "greenish"). Kawakami et al (2016) build character-level models for predicting colors given descriptions in addition to describing colors. Their model uses a Labspace color representation and uses the color to initialize the LSTM instead of feeding it in at each time step; they also focus on visualizing point predictions of their description-to-color model, whereas we examine the full distributions implied by our color-todescription model.…”

Section: Discussionmentioning

confidence: 99%

Learning to Generate Compositional Color Descriptions

Monroe

Goodman

Potts

2016

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

The production of color language is essential for grounded language generation. Color descriptions have many challenging properties: they can be vague, compositionally complex, and denotationally rich. We present an effective approach to generating color descriptions using recurrent neural networks and a Fouriertransformed color representation. Our model outperforms previous work on a conditional language modeling task over a large corpus of naturalistic color descriptions. In addition, probing the model's output reveals that it can accurately produce not only basic color terms but also descriptors with non-convex denotations ("greenish"), bare modifiers ("bright", "dull"), and compositional phrases ("faded teal") not seen in training.

show abstract

“…In the ESI, † the details of the survey are given. Note that in contrast to other works 23,30,33,34 based on the xkcd survey we did not attempt to build a general model that maps colour names to the tristimulus coordinates of the intended colour but rather want to infer the likelihood of the intended colour for all colour names that are used for MOFs in the CSD.…”

Section: Colours and Their Perceptionmentioning

confidence: 99%