‘Lighter’ Can Still Be Dark: Modeling Comparative Color Descriptions

Abstract

We propose a novel paradigm of grounding comparative adjectives within the realm of color descriptions. Given a reference RGB color and a comparative term (e.g., 'lighter', 'darker'), our model learns to ground the comparative as a direction in the RGB space such that the colors along the vector, rooted at the reference color, satisfy the comparison. Our model generates grounded representations of comparative adjectives with an average accuracy of 0.65 cosine similarity to the desired direction of change. These vectors approach colors with Delta-E scores of under 7 compared to the target colors, indicating the differences are very small with respect to human perception. Our approach makes use of a newly created dataset for this task derived from existing labeled color data.

How Large Are Lions? Inducing Distributions over Quantitative Attributes

“…Our work overlaps with a number of approaches to ground textual objects by: achieving a commonsense understanding of numeric expressions (Chaganty and Liang, 2016), grounding adjectives into RGB colors (Winn and Muresan, 2018), grounding events duration (Pan et al, 2006;Gusev et al, 2011) and measurements' intensity within a given context (Narisawa et al, 2013).…”

Grounding learning of modifier dynamics: An application to color naming

“…The dataset 4 used to train and evaluate our model includes 415 triples (reference color label, r, modifier, m, and target color label, t) in RGB space presented by Winn and Muresan (2018). Munroe (2010) collected the original dataset consisting of color description pairs collected in an open online survey; the dataset was subsequently filtered by McMahan and Stone (2015).…”

“…We train models in both RGB and HSV color space, but samples in WM18 are only presented in RGB space. Because modifiers encode the general relationship between r and t we use the same approach presented by Winn and Muresan (2018): using the mean value of a set of points to represent a color. A drawback of this approach is that it does not account for our uncertainty about the appropriate RGB encoding for a given color word.…”

“…Model configuration: The model presented by Winn and Muresan (2018) is initialized with Google's pretrained 300-d word2vec embeddings (Mikolov et al, 2013b,a) which are not updated during training. To perform comparable experiments, all models in paper are designed with the same pre-trained embedding model.…”

“…For example, as shown in Figure 1, given a reference color code r = 101 55 0 and a modifier m = "greenish", our models are trained to predict the target color code t = 105 97 18 . The state-of-the-art approach for this task (Winn and Muresan, 2018) represents both colors and modifiers as vectors in RGB space, and learns a 1 Code available at https://github.com/ HanXudong/GLoM Figure 1: Examples of the grounded modifier modelling task, shown in RGB space. Given the reference and modifier, the system must predict the target color.…”