We reported a new result of the neutral impurity scattering of holes that has impact on the charge drift mobility in high purity p-type germanium crystals at 77 Kelvin. The charge carrier concentration, mobility and resistivity are measured by Hall Effect system at 77 Kelvin. We investigated the contribution to the total charge drift mobility from ionized impurity scattering, lattice scattering, and neutral impurity scattering with the best theoretical models and experimental data. Several samples with measured Hall mobility from the grown crystals are used for this investigation. With the measured Hall mobility and ionized impurity concentration as well as the theoretical models, we calculated the neutral impurity concentration by the Matthiessen's rule. As a result, the distributions of the neutral impurity concentrations with respect to the radius of the crystals are obtained. Consequently, we demonstrated that neutral impurity scattering is a significant contribution to the charge drift mobility, which has dependence on the concentration of neutral impurities in a given germanium crystal.