Coherence in the Visual Imagination

Abstract

An incoherent visualization is when aspects of different senses of a word (e.g., the biological "mouse" vs. the computer "mouse") are present in the same visualization (e.g., a visualization of a biological mouse in the same image with a computer tower). We describe and implement a new model of creating contextual coherence in the visual imagination called Coherencer, based on the SOILIE model of imagination. We show that Coherencer is able to generate scene descriptions that are more coherent than SOILIE's original approach as well as a parallel connectionist algorithm that is considered competitive in the literature on general coherence. We also show that co-occurrence probabilities are a better association representation than holographic vectors and that better models of coherence improve the resulting output independent of the association type that is used. Theoretically, we show that Coherencer is consistent with other models of cognitive generation. In particular, Coherencer is a similar, but more cognitively plausible model than the C model of concept combination created by Costello and Keane (2000). We show that Coherencer is also consistent with both the modal schematic indices of perceptual symbol systems theory (Barsalou, 1999) and the amodal contextual constraints of Thagard's (2002) theory of coherence. Finally, we describe how Coherencer is consistent with contemporary research on the hippocampus, and we show evidence that the process of making a visualization coherent is serial.

Do We Need Neural Models to Explain Human Judgments of Acceptability?

“…Measures of semantic coherence are used in models of document-level topic generation (Mimno, Wallach, Talley, Leenders, & McCallum, 2011) and visual scene generation (Vertolli, Kelly, & Davies, 2018) to ensure, respectively, that the topics and scenes "make sense." We hypothesize that more acceptable sentences have higher semantic coherence.…”