The semi‐empirical MT_CKD model of the absorption continuum of water vapor is widely used in atmospheric radiative transfer codes of the atmosphere of Earth and, recently, of exoplanets but lacks of experimental validation in the atmospheric windows. Recent laboratory measurements by Fourier transform Spectroscopy led to self‐continuum cross sections much larger than the MT_CKD values in the near‐infrared transparency windows. We report on accurate measurements by Cavity Ring Down Spectroscopy (CRDS) and Optical Feedback‐Cavity‐Enhanced Absorption Spectroscopy (OF‐CEAS) at selected spectral points of the transparency windows centered around 4.0, 2.1, and 1.25 µm. The temperature dependence of the absorption continuum at 4.38 µm is measured in the 23–39°C range. The self‐continuum water vapor absorption is derived either from the baseline variation of water vapor spectra recorded for a series of pressure values over a small spectral interval or from baseline monitoring at fixed laser frequency during pressure ramps. After subtraction of the local water monomer lines contribution, self‐continuum cross sections, Cs, are determined with an accuracy better than 10% from the pressure squared dependence of the continuum absorption measured up to about 15 Torr. Together with our previous measurements in the 2.1 and 1.6 µm windows, the present results provide a unique set of water vapor cross sections for testing the MT_CKD model in four transparency windows. Although showing some important deviations of the absolute values (up to about a factor 4), our accurate measurements validate the overall frequency dependence of the most recent version (V2.8) of the MT_CKD model.