We have previously presented broadband variable transmission sphere (BVTS) systems based on the Alvarez lens for use in Fizeau interferometry. A BVTS system may replace multiple conventional transmission spheres with one system composed of two plano-freeform optical elements. In this paper, we discuss retrace error as a source of aberrations in Fizeau interferometry measurements when using BVTS systems. We explore a design optimization method to decrease retrace error by optimizing freeform surfaces to reduce targeted Fringe Zernike wavefront aberrations. Significant reduction in peak-to-valley retrace error is demonstrated.