Interior permanent magnet synchronous motors (IPMSMs) have been widely used in a variety of applications. The IPMSM will generate vibration as well as functioning like other traditional motors, which will affect the performance and the durability of the motor itself. The modal and vibration characteristics of the IPMSM and an optimization design to reduce vibration are researched in this paper. The formulas of electromagnetic force are deduced and analyzed, and a finite element model is established. The model is used for modal calculation, and the low order natural frequency is obtained. Then, the modal test is carried out by the hammering method, and the resonance frequency is found by analyzing the results. Eventually, the electromagnetic vibration is analyzed by taking the electromagnetic force as the load condition, the vibration displacement waveform is obtained, and the correctness of the dynamic calculation is verified by vibration tests. In addition, an eccentric tooth edge design is proposed to weaken the dominant radial electromagnetic force and the vibration displacement is compared with the designed prototype. This design method, which could help to reduce the vibration and noise of the IPMSM, has great potential in future applications.