We apply a diagrammatic Monte Carlo method to the problem of an impurity interacting resonantly with a homogeneous Fermi bath for a quasi-two-dimensional setup. Notwithstanding the series divergence, we can show numerically that the three particle-hole diagrammatic contributions are not contributing significantly to the final answer, thus demonstrating a nearly perfect destructive interference of contributions in subspaces with higher-order particle-hole lines. Consequently, for strong enough confinement in the third direction, the transition between the polaron and the molecule ground state is found to be in good agreement with the pure two-dimensional case and agrees very well with the one found by the wave-function approach in the two-particle-hole subspace.