We fabricated a Y2O3–ZrO2 film (YZO) on Al2O3 to achieve the field effect passivation with high negative fixed charge densities on p-type Si. The surface recombination velocity was improved down to 30 cm/s after annealing at 400 °C. This improvement can be attributed to the effective fixed charge enhancement while the interface state densities were kept almost constant. A high thermal tolerance of over 600 °C upon inserting a 2-nm-thick ZrO2 layer between the YZO and Al2O3 interface was confirmed. This result showed that the ZrO2 layer acts as a protective barrier to prevent Al and Y interdiffusions. Annealing at a higher temperature of 800 °C resulted in interface degradation and YZO crystallization, which led to the deterioration of the passivation properties.