This work addresses vibrational energy harvesting using magnetostrictive materials. In this field, materials with exceptional magneto-mechanical coupling properties (e.g., Galfenol, Terfenol-D) have attracted significant attention. Only a few magnetostrictive materials have been tested in devices, however, leaving the actual influence of these materials’ properties on the energy harvesting device open to question. This work compares an extensive range of ferromagnetic materials through analysis of their magnetic behavior under static stress. To enable fair comparison of the materials, a model was developed to interpolate their magnetic anhysteretic curves under fixed stress of σ = ±50 MPa. The energy harvesting process was then simulated using a theoretical Ericsson thermodynamic cycle, where the area represents the energy density. This approach estimates the ultimate energy density of the materials using a fair approach, without placing conditions on the applied magnetic field. The correlation between ultimate energy density and the magnetoelastic coefficient show that highly magnetostrictive materials achieve higher ultimate energy densities, as expected. In the low field range, it is however concluded that all materials exhibit energy densities of the same order of magnitude. Secondly, the magnetoelastic coefficient versus excitation field characteristics revealed an optimal bias magnetic field for each material. Finally, for realistic implementation, the paper considers a pre-stress in combination with a bias magnetic field and the small dynamic variations that result from currents induced in surrounding coils. A model was developed and revealed an optimum output energy density that was independent of the geometry and the coil. An energy harvesting figure of merit was then defined to enable a final comparison of the materials, encompassing both material characteristics and realistic applications. Under these working conditions and with all costs considered, some low-magnetostriction materials appeared able to compete with giant magnetostriction materials.