We consider the interplay of the Early Dark Energy (EDE) model, the Swampland Distance Conjecture (SDC), and cosmological parameter tensions. EDE is a proposed resolution of the Hubble tension relying upon a near-Planckian scalar field excursion, while the SDC predicts an exponential sensitivity of masses of other fields to such an excursion, m ∝ e −c|∆φ|/M pl with c ∼ O(1). Meanwhile, EDE is in tension with large-scale structure (LSS) data, due to shifts in the standard ΛCDM parameters necessary to fit the cosmic microwave background (CMB). One might hope that a proper treatment of the model, e.g., accounting for the SDC, may ameliorate the tension with LSS. Motivated by these considerations, we introduce the Early Dark Sector (EDS) model, wherein the mass of dark matter is exponentially sensitive to super-Planckian field excursions of the EDE scalar. The EDS model exhibits new phenomenology in both the early and late universe, the latter due to an EDE-mediated dark matter self-interaction, which manifests as an enhanced gravitational constant on small scales. This EDE-induced dark matter-philic "fifth force", while constrained to be small, remains active in the late universe and is not screened in virialized halos. We find that the new interaction with dark matter partially resolves the LSS tension. However, the marginalized posteriors are nonetheless consistent with fEDE = 0 at 95% CL once the Dark Energy Survey Year 3 measurement of S8 is included. We additionally study constraints on the model from Atacama Cosmology Telescope data, and find a factor of two improvement on the error bar on the SDC parameter c, along with an increased preference for the EDE component. We discuss the implications of these constraints for the SDC, and find the tightest observational constraints to date on a swampland parameter, suggesting that an EDE description of cosmological data is in tension with the SDC.