We show that every word hyperbolic, surface-by-(noncyclic) free group Γ is as rigid as possible: the quasi-isometry group of Γ equals the abstract commensurator group Comm(Γ), which in turn contains Γ as a finite index subgroup. As a corollary, two such groups are quasi-isometric if and only if they are commensurable, and any finitely generated group quasi-isometric to Γ must be weakly commensurable with Γ. We use quasi-isometries to compute Comm(Γ) explicitly, an example of how quasi-isometries can actually detect finite index information. The proofs of these theorems involve ideas from coarse topology, Teichmüller geometry, pseudo-Anosov dynamics, and singular solv geometry.