The SU(r ) Vafa-Witten partition function, which virtually counts Higgs pairs on a projective surface S, was mathematically defined by Tanaka-Thomas. On the Langlands dual side, the first-named author recently introduced virtual counts of Higgs pairs on μ r -gerbes. In this paper, we instead use Yoshioka's moduli spaces of twisted sheaves. Using Chern character twisted by rational B-field, we give a new mathematical definition of the SU(r )/Z r Vafa-Witten partition function when r is prime. Our definition uses the period-index theorem of de Jong. S-duality, a concept from physics, predicts that the SU(r ) and SU(r )/Z r partition functions are related by a modular transformation. We turn this into a mathematical conjecture, which we prove for all K 3 surfaces and prime numbers r .Communicated by Jean-Yves Welschinger.