The virial expansion characterizes the high-temperature approach to the quantum-classical crossover in any quantum many-body system. Here, we calculate the virial coefficients up to the fifth order of Fermi gases in one, two, and three dimensions, with attractive contact interactions, as relevant for a variety of applications in atomic and nuclear physics. To that end, we discretize the imaginary-time direction and calculate the relevant canonical partition functions. In coarse discretizations, we obtain analytic results featuring relationships between the interaction-induced changes b 3 , b 4 , and b 5 as functions of b 2 , the latter being exactly known in many cases by virtue of the Beth-Uhlenbeck formula. Using automated-algebra methods, we push our calculations to progressively finer discretizations and extrapolate to the continuous-time limit. We find excellent agreement for b 3 with previous calculations in all dimensions and we formulate predictions for b 4 and b 5 in one and two dimensions. We also provide, for a range of couplings, the subspace contributions b 31 , b 22 , b 41 , and b 32 , which determine the equation of state and static response of polarized systems at high temperature. As a performance check, we compare the density equation of state and Tan contact with quantum Monte Carlo calculations, diagrammatic approaches, and experimental data where available. Finally, we apply Padé and Padé-Borel resummation methods to extend the usefulness of the virial coefficients to approach and in some cases go beyond the unit-fugacity point.