Recent experiments have measured the signatures of the Kondo effect in the zero-field thermopower of strongly correlated quantum dots [Svilans et al., Phys. Rev. Lett. 121, 206801 (2018); Dutta et al., Nano Lett. 19, 506 (2019)]. They confirm the predicted Kondo-induced sign change in the thermopower, upon increasing the temperature through a gate-voltage dependent value T 1 T K , where T K is the Kondo temperature. Here, we use the numerical renormalization group (NRG) method to investigate the effect of a finite magnetic field B on the thermopower of such quantum dots. We show that, for fields B exceeding a gate-voltage dependent value B 0 , an additional sign change takes place in the Kondo regime at a temperature T 0 (B ≥ B 0 ) > 0 with T 0 < T 1 . The field B 0 is comparable to, but larger than, the field B c at which the zero-temperature spectral function splits in a magnetic field. The validity of the NRG results for B 0 are checked by comparison with asymptotically exact higher-order Fermi-liquid calculations [Oguri et al., Phys. Rev. B 97, 035435 (2018)]. Our calculations clarify the field-dependent signatures of the Kondo effect in the thermopower of Kondo-correlated quantum dots and explain the recently measured trends in the B-field dependence of the thermoelectric response of such systems [Svilans et al., Phys. Rev. Lett. 121, 206801 (2018)]. K ≈ 0.50 where T HWHM K is the HWHM of the T = 0 Kondo resonance given by T HWHM K