For G = GL2, PGL2, SL2 we prove that the perverse filtration associated with the Hitchin map on the rational cohomology of the moduli space of twisted G-Higgs bundles on a compact Riemann surface C agrees with the weight filtration on the rational cohomology of the twisted G character variety of C when the cohomologies are identified via non-Abelian Hodge theory. The proof is accomplished by means of a study of the topology of the Hitchin map over the locus of integral spectral curves.