Objectively quantifying subjective phenomena like visual illusions is challenging. We address this issue in the context of the Flashed Face Distortion Effect (FFDE), where faces presented in succession appear distorted and grotesque. We first show that the traditional method of quantifying FFDE – via subjective ratings of the level of distortion – is subject to substantial biases. Motivated by this finding, we develop an objective method for quantifying FFDE by introducing two design innovations. First, we create artificially distorted faces and ask observers to discriminate between undistorted and objectively distorted faces. Second, we employ both an illusion condition, which includes a succession of 15 face flashes, and a control condition, which includes a single face flash and does not induce an illusion. Using these innovations, we quantify the strength of the FFDE illusion by comparing the response bias for identifying distorted faces between the illusion and control conditions. We find that our method successfully quantifies the face distortion, with observers exhibiting a more liberal response bias in the illusion condition. Finally, we apply our new method to evaluate how the FFDE illusion is modulated by face eccentricity, face inversion, the temporal frequency of the face flashes, and presence of temporal gaps between consecutive faces. Our results demonstrate the utility of our objective method in quantifying the subjective illusion of face distortion. Critically, the method is general and can be applied to other phenomena that are inherently subjective.