The structural determination of clathrate hydrates, nonstoichiometric crystalline host-guest materials, is challenging because of the dynamical disorder and partial cage occupancies of the guest molecules. The application of direct space methods with Rietveld analysis can determine the powder X-ray diffraction (PXRD) patterns of clathrates. Here, we conducted Rietveld analysis with the direct space method for the structural determination of binary tetrahydrofuran (THF) + O2 and 3-hydroxytetrahydrofuran (3-OH THF) + O2 clathrate hydrates in order to identify the hydroxyl substituent effect on interactions between the host framework and the cyclic ether guest molecules. The refined PXRD results reveal that the hydroxyl groups are hydrogen-bonded to host hexagonal rings of water molecules in the 51264 cage, while any evidences of hydrogen bonding between THF guests and the host framework were not observed from PXRD at 100 K. This guest-host hydrogen bonding is thought to induce slightly larger 512 cages in the 3-OH THF hydrate than those in the THF hydrate. Consequently, the disorder dynamics of the secondary guest molecules also can be affected by the hydrogen bonding of larger guest molecules. The structural information of binary clathrate hydrates reported here can improve the understanding of the host-guest interactions occurring in clathrate hydrates and the specialized methodologies for crystal structure determination of clathrate hydrates.