Zircaloy-4 plates of 2 mm in thickness were successfully joined by friction stir welding with rapid cooling. The processing temperatures were strictly controlled below the α/β phase transformation point. The stir zone (SZ) has an ultra-refined grain structure with high dislocation density and abundant twin boundaries. Grain subdivision and discontinuously dynamic recrystallisation contributed most to the grain refinement. A good match of strength and uniform elongation of 895 MPa and 16.8%, respectively, was achieved in the SZ by using liquid nitrogen cooling. This work provides an effective strategy to enhance the strength of friction stir welded Zircaloy-4 joint without ductility loss.