Infrared solar spectrum observed by ground-based telescopes is seriously affected by the background radiation both from the telescope and sky, relative to the visible wavelength. Its accuracy is also influenced by the spectral resolution of the Fourier transform spectrometer. In the paper, we developed a $CO_2$ gas cell and installed it in the sample compartment to calibrate the spectral resolution of the Bruker IFS-125HR at the infrared wavelength. The measured spectral resolution is \textbf{ 0.00342 $\pm 0.00086 \, cm^{-1}$ and 0.0059 $ \pm 0.00024 \, cm^{-1}$ at the wavenumber of 798 $cm^{-1}$ and 2136 $cm^{-1}$, respectively}. We also updated a fully reflected sun-light feeding system to observe the solar spectrum near the CO 4.66 $\mu m$ and Mg I 12.32 $\mu m$. By quickly pointing the sun-light feeding system about 1 degree away from the solar disk center, we are able to measure the background radiation from the telescope and the sky at Huairou Solar Observing Station. After removing the background radiation, our observed solar spectrum at CO 4.66 $\mu m$ is consistent with that from National Solar Observatory. The Mg I 12.32 $\mu m$ working line selected by the Accurate Infrared Magnetic Field Measurements of the Sun (AIMS) project is also identified. Our method is helpful not only for the spectral resolution calibration and background radiation correction of AIMS but also for other infrared astronomical telescopes.