The purpose of this article is to review and bring new contributions in understanding the relationship between molecular/supramolecular architecture at the nanometer scale and macroscopic mechanical properties in polyurethanes. In this article, this problem is addressed by studying numerous conventional segmented polyurethanes, based on several diisocyanates, macrodiols, and chain extenders. In order to widen the range of structures achievable beyond those normally available, in this review, we have also included recent developments that have been made in polyurethane materials by producing polymers based on isocyanates of variable geometry, giving hard segments that allow the variation of hard‐domain crystallinity as a key structural variable. They were used to probe the sensitivity of inelastic effects to structural details of the materials. Mechanical responses studied were large strain, constant strain rate, and cyclic strain responses, including interrupted tests. Results were related to microstructural changes, on the basis of evidence from small‐angle X‐ray scattering (SAXS) and wide‐angle X‐ray scattering (WAXS) and infrared (IR) dichroic measurements. Inelastic effects were most pronounced when the hard segment crystallized, and when the phase segregation was more pronounced. Thermal techniques such as differential scanning calorimetry (DSC) and dynamic mechanical analysis (DMA) were used to study the thermal behavior of the materials. There were tendencies to phase separation, with a characteristic length of ca. 20 nm, and, when isocyanates of variable geometry were employed with certain chain extenders, to crystallization of the hard phase. Such materials display higher flow stress in the hard phase caused by stronger phase segregation. This has clearly revealed that inelastic features in the constitutive response of polyurethane elastomers are sensitive to microstructural detail on length scales of the order of nanometers. Hard‐domain crystallinity exerts a strong effect on inelasticity of the elastomers. There is a clear indication that the physical origin of the flow stress is the relative displacement of the hydrogen‐bonded hard segments. Inelasticity, as measured by hysteresis and unrecovered strain, was increased by increased phase separation.