Context. The hydroxyl radical (OH) is found in various environments within the interstellar medium (ISM) of the Milky Way and external galaxies, mostly either in diffuse interstellar clouds or in the warm, dense environments of newly formed low-mass and high-mass stars, i.e., in the dense shells of compact and ultracompact HII regions (UCHIIRs). Until today, most studies of interstellar OH involved the molecule's radio wavelength hyperfine structure (hfs) transitions. These lines are generally not in local thermodynamical equilibrium (LTE) and either masing or over-cooling complicates their interpretation. In the past, observations of transitions between different rotational levels of OH, which are at far-infrared (FIR) wavelengths, have suffered from limited spectral and angular resolution. Since these lines have critical densities many orders of magnitude higher than the radio wavelength ground state hfs lines and are emitted from levels with more than 100 K above the ground state, when observed in emission, they probe very dense and warm material. Aims. We aim to probe the warm and dense molecular material surrounding the UCHIIR/OH maser sources W3(OH), G10.62−0.39 and NGC 7538 IRS1 by studying the 2 Π 1/2 , J = 3/2−1/2 rotational transition of OH in emission and, toward the last source also the molecule's 2 Π 3/2 , J = 5/2−3/2 ground-state transition in absorption. Methods. We used the Stratospheric Observatory For Infrared Astronomy (SOFIA) to observe these OH lines, which are near 1.84 THz (163 μm) and 2.51 THz (119.3 μm), with high angular (∼16 /11 ) and spectral resolution (better than 1 km s −1 ). Results. We clearly detect the OH lines, some of which are blended with each other. Employing non-LTE radiative transfer calculations we predict line intensities using models of a low OH abundance envelope versus a compact, high-abundance source corresponding to the origin of the radio OH lines. From the observed velocities and line-widths we can place constraints on the origin of the emission, and with detailed modeling we show for instance that the OH emission of W3(OH) comes from the UCHIIR and not from the envelope of the nearby hot-core. Conclusions. The FIR lines of the OH molecule provide important information on the density and temperature structure of UCHIIRs. Taking a low-abundance envelope component is not sufficient to reproduce the spectra for W3(OH) and G10.62−0.39, a compact, high OH column density source -corresponding to the OH radio emitting sources is definitively required.