The large (0.6-1 AU) and intermediate-scale (1.5-100 Mm) structure, processes and properties of the solar wind (SW), such as heliospheric current sheet (HCS) dynamics, coronal mass ejections (CMEs), stream interaction (or co-rotating interaction) regions (SIRs), and interplanetary (IP) shocks, have a crucial impact on particle energization and propagation, as well as on the evolution of various magnetospheric processes. However, their detailed evolution in the inner heliosphere has not been adequately investigated yet, due to the lack of simultaneous, multi-point, coordinated in-situ observations at appropriate length-scales. As is takes about ≈ 1 h with typical SW speeds to reach Earth from Earth-Sun L1 (EL-1) point, the EL-1 measurements cannot provide sufficient warning time for space weather operations. Furthermore, if this structure is present in the SW [10], single spacecraft (SC) measurements at EL-1 will lead to inaccuracies when forecasting or developing various space weather warnings, e.g., for radiation belt electron flux enhancements etc. Missing this structure would also lead to wrong conclusions when interpreting the onset conditions of the physical processes at the magnetopause [51, 10], and in statistical magnetospheric studies that use lagged observations from a single monitor at EL-1 for characterizing SW properties and interplanetary magnetic field (IMF) orientation at the bow-shock nose. Thus, the goal of this white paper is to discuss the physical processes and structures in the SW which hinder accurate space weather forecasting and therefore motivate new, multi-point and multi-scale, critical measurements to improve our knowledge of the 3-D, large to intermediate-scale structures in the SW, azimuthal variation, radial evolution, as well as their impact on the charged particle acceleration, heating, and transport in the inner heliosphere. This science whitepaper also motivates our Seven Sisters inner heliospheric mission concept (see our companion mission concept white-paper), which is designed to solve these science questions.