The role of solvent composition (polar minor components, PC) on β‐sitosterol/γ‐oryzanol oleogel formation, appearance, and properties is studied. Solvent composition is altered via thermal treatment and the elimination of natural PC from untreated sunflower and canola oil. A maximum in oleogel hardness is found. DSC reveals that sol‐gel transition is increasingly suppressed with the level of PC. This is attributed to two distinct mechanisms: enlarged interactions of PC with the structuring elements and a reduction in diffusion due to a higher solvent viscosity. Gel‐sol‐transition consists of two concurrent processes: decomposition of tube bundles (peak fronting) and the dissolution of tubes (peak). The peak area decreases with increasing solvent permittivity while simultaneously the area of the fronting increases. Gel‐sol‐transition temperatures are practically invariable for all samples indicating that the maximum in gel hardness cannot be related to changes in the solubility of the sterols. AFM microscopy reveals that the arrangement of the elements of scaffolding changes considerably with the level of PC. This work demonstrates the strong impact of PC on the self‐assembly of β‐sitosterol/γ‐oryzanol and oleogel properties. A detailed characterization of the oils is thus inevitable to perform trustworthy research. Practical Applications: It is widely known that a high intake of saturated fatty acids increases the risk of suffering from cardiovascular diseases. Unfortunately, their ability to provide unique texture to food products can hardly be met. Oleogelation has the potential to deliver the solid structure necessary for various fat‐based food products by transferring an oil rich in essential fatty acids into a solid‐like structure. Moreover, the nutritional value of these oils remains nearly unchanged. It is found that oil composition, precisely the fatty acid composition of the TAGs and minor oil components have a profound impact on oleogel properties. Fundamental understanding of network properties and formation of oleogels helps to maximize their capability for industrial application. The work presented reveals that detailed documentation of the quality of the oils, in particular the fatty acid composition and presence of polar minor components, is a necessary prerequisite to conduct reliable scientific work in the field oleogel research. Hardness and spatial arrangement of network building blocks of of β‐sitosterol/γ‐oryzanol oleogels is modified in the presence/absence of polar minor components.