We derive a holomorphic anomaly equation for the Vafa-Witten
partition function for twisted four-dimensional
\mathcal{N} =4𝒩=4
super Yang-Mills theory on \mathbb{CP}^{2}ℂℙ2
for the gauge group SO(3)SO(3)
from the path integral of the effective theory on the Coulomb branch.
The holomorphic kernel of this equation, which receives contributions
only from the instantons, is not modular but ‘mock modular’. The
partition function has correct modular properties expected from
SS-duality
only after including the anomalous nonholomorphic boundary contributions
from anti-instantons. Using M-theory duality, we relate this phenomenon
to the holomorphic anomaly of the elliptic genus of a two-dimensional
noncompact sigma model and compute it independently in two dimensions.
The anomaly both in four and in two dimensions can be traced to a
topological term in the effective action of six-dimensional
(2,0)(2,0)
theory on the tensor branch. We consider generalizations to other
manifolds and other gauge groups to show that mock modularity is generic
and essential for exhibiting duality when the relevant field space is
noncompact.