Magnetic shape memory materials require a high twin boundary mobility and low hysteresis for applications mainly as actuators, sensors, and magnetocaloric cooling elements. Usually, outstanding properties are found only in samples with a modulated martensitic structure. Here, we analyze the question why a modulated structure is beneficial and show evidence that the modulated martensite is not an equilibrium phase but a nanoscale microstructure of non‐modulated (NM) martensite. In this review, we combine results from continuum and atomistic theory, as well as local and integral measurements on the model system Ni–Mn–Ga. Following the concept of adaptive martensite the modulated phase forms to minimize elastic energy near the phase boundary by introducing low‐energy twin boundaries between lamellae of the NM martensite that have widths of a few unit cells.