We lay the foundations for the construction of analytic expressions for Fourier-domain gravitational waveforms produced by eccentric, inspiraling compact binaries in a post-circular or smalleccentricity approximation. The time-dependent, "plus" and "cross" polarizations are expanded in Bessel functions, which are then self-consistently re-expanded in a power series about zero initial eccentricity to eighth order. The stationary phase approximation is then employed to obtain explicit analytic expressions for the Fourier transform of the post-circular expanded, time-domain signal. We exemplify this framework by considering Newtonian-accurate waveforms, which in the post-circular scheme give rise to higher harmonics of the orbital phase and amplitude corrections both to the amplitude and the phase of the Fourier domain waveform. Such higher harmonics lead to an effective increase in the inspiral mass reach of a detector as a function of the binary's eccentricity e0 at the time when the binary enters the detector sensitivity band. Using the largest initial eccentricity allowed by our approximations (e0 < 0.4), the mass reach is found to be enhanced up to factors of approximately 5 relative to that of circular binaries for Advanced LIGO, LISA, and the proposed Einstein Telescope at a signal-to-noise ratio of ten. A post-Newtonian generalization of the post-circular scheme is also discussed, which holds the promise to provide "ready-to-use" Fourier-domain waveforms for data analysis of eccentric inspirals.