The accuracy of neutron scattering cross sections is the gauge for the realistic outcome of a neutron transport simulation. To improve the traditional harmonic physics model used in such simulations, we revisit the slow neutron transport theory in crystalline materials and aim to develop a unified model that has good performance for neutron transport problems in crystals in a wide range of temperatures and pressures. The quasi-harmonic approximation (QHA) correlates phonon evolution explicitly with unit cell volume. Therefore, it is capable of evaluating a variety of material properties at finite temperatures. In this work, we show numerically that it is a very effective tool for our application as well. Within the framework of QHA, we calculate the temperature dependent characteristics of phonons in three elemental crystals, namely Be, Mg and Al. Based on the obtained results, our calculated neutron total cross sections agree closely with experimental transmission cross sections in a large temperature range below the melting point. We show that as the harmonic cross section model ignores the effects of phonon softening in these crystals, it underestimates the total inelastic cross sections at high temperatures. In the case of Al, we observe that such underestimation is up to 7% at room temperature. In addition, we study the phonon-phonon scatterings in Al. We observe that the cross section is insensitive to the finite phonon lifetimes even at 800 K.