Thermal displacement fluctuations in a crystal may be classified as either "affine" or "nonaffine". While the former couples to external stress with familiar consequences, the response of a crystal when non-affine displacements are enhanced using the thermodynamically conjugate field, is relatively less studied. We examine this using a simple model of a crystal in two dimensions for which analytical calculations are possible. Enhancing non-affine fluctuations destabilises the crystal. The population of small frequency phonon modes increases, with the phonon density of states shifting, as a whole, towards zero frequency. Even though the crystal is free of disorder, we observe growing length and time scales. Our results, which may have implications for the glass transition and structural phase transitions in solids, are compared to molecular dynamics simulations. Possibility of experimental verification of these results is also discussed.