For mechanical reinforcement against the huge Lorentz forces in high magnetic fields, epoxy resins are commonly used to impregnate high temperature superconductor (HTS) coils. Due to the different thermal expansion coefficients between the epoxy resin and the HTS tape, the contraction of the epoxy resin causes critical current degradation in the HTS coils during the cooling process. Alternative techniques, such as coils with no insulation, can avoid current degradation, however their structure has no reinforcement between layers. In view of this, we tried to apply the ice impregnation technique of pulsed magnets to HTS coils for the first time. HTS coils impregnated with ice could avoid critical current degradation after thermal cycles. Also, local heat pulse tests and overcurrent tests showed that the ice impregnated coil had better thermal stability than the epoxy impregnated coil. Moreover, the fabrication process of the ice impregnated coil is simple and economical. Meanwhile HTS tapes can be reused after impregnation. The ice impregnation technique is a promising prospect for high temperature superconducting coils.