Magnetic nanoparticles (MNPs) have been extensively explored as magnetic resonance imaging (MRI) contrast agents. With the increasing complexity in the structure of modern MNPs, the classical Solomon-Bloembergen-Morgan and the outer-sphere quantum mechanical theories established on simplistic models have encountered limitations for defining the emergent phenomena of relaxation enhancement in MRI. We reviewed recent progress in probing MRI relaxivity of MNPs based on structural features at the molecular and atomic scales, namely the structure-relaxivity relationships, including size, shape, crystal structure, surface modification, and assembled structure. We placed a special emphasis on bridging the gaps between classical simplistic models and modern MNPs with elegant structural complexity. In the pursuit of novel MRI contrast agents, we hope this review will spur the critical thinking for design and engineering of novel MNPs for MRI applications across a broad spectrum of research fields.