The Invariance Hypothesis was originally proposed by George Lakoff and Mark Turner in 1989. Since then, a range of versions has evolved so that there are currently both strong and weak statements of it. In general, the Invariance Hypothesis suggests a constraint on the information carried in a metaphorical mapping, as modelled in cognitive linguistics. It seeks to preserve the receiver’s knowledge about the target domain of a metaphor, so that the target retains its basic conceptual integrity in the mapping process. In other words, only that amount of the source domain that is consistent with the preservation of the target is mapped. Invariance is proposed to resolve a perceived problem in accounting for some metaphors, in order to sustain the claims of cognitive linguistics to be a useful and applicable model of language. However, I believe that this is mistaken, and that acceptance of the Invariance Hypothesis is itself a threat to the value of cognitive linguistics, as applied to literature (where it has come to be called cognitive poetics or cognitive stylistics). I will use literary examples to argue for the rejection of the Invariance Hypothesis, which curtails the perception of metaphor as creative, and cannot explain its capacity for reference to a new sense beyond source and target. This limitation is counter to the larger claims of cognitive linguistics concerning the linguistic basis and embodiment of culture and perception. Finally, I suggest an alternative solution, arising from the analysis of literary examples, which preserves the general value of cognitive linguistics while escaping the inflexibility of invariance.