We investigate the in-medium modifications of heavy quarkonia in the vector channel and the heavy quark diffusion coefficient by comparing Euclidean correlators from the lattice to a perturbative spectral function. On the lattice side, we work with continuum extrapolated data from four different large and fine lattices with Clover-improved Wilson fermions in the quenched approximation at five temperatures (0.75, 1.1, 1.3, 1.5 and 2.25T c ). On the perturbative side, we use a combination of pNRQCD and vacuum asymptotics to describe the spectral function. After accounting for systematic errors, we obtain a spectral function that is suited to describe the bound state region. This spectral function describes charmonium well without a resonance peak at any of our analyzed temperatures above T c , while we observe a thermally broadened resonance peak for bottomonium that is only melted at our largest temperature, 2.25T c . For the transport contribution we assume a Breit-Wigner shaped peak and find that the drag coefficient of charm quarks is larger than that of bottom quarks.