A proof-of-concept study is undertaken to demonstrate the utility of the machine learning combined with the thermodynamic perturbation theory (MLPT) to test the accuracy of electronic structure methods in finite-temperature thermodynamic calculations. As a test example, formic acid dimer is chosen, which is one of the systems included in the popular benchmark set S22 [Jurečka et al., Phys. Chem. Chem. Phys. 8, 1985–1993 (2006)]. Starting from the explicit molecular dynamics and thermodynamic integration performed at the PBE + D2 level, the MLPT is used to obtain fully anharmonic dimerization free and internal energies at the reference quality CCSD(T) level and 19 different density functional approximations, including GGA, meta-GGA, non-local, and hybrid functionals with and without dispersion corrections. Our finite-temperature results are shown to be both qualitatively and quantitatively different from those obtained using the conventional benchmarking strategy based on fixed structures. The hybrid functional HSE06 is identified as the best performing approximate method tested, with the errors in free and internal energies of dimerization being 36 and 41 meV, respectively.