Do you know the Wooly Bully? Testing Cultural Knowledge to Verify Participant Age Online

Abstract

People in online studies sometimes misrepresent themselves. Regardless of their motive for doing so, participant misrepresentation threatens the validity of research. Here, we propose and evaluate a way to verify the age of online respondents: a test of cultural knowledge. Across six studies (N = 1,543), participants of various ages completed an age verification instrument. The instrument assessed familiarity with cultural phenomena (e.g., songs and tv shows) from decades past and present. We consistently found our instrument discriminated between people of different ages. In Studies 1a and 1b, age strongly correlated with performance on the instrument (mean r = .8). In Study 2, the instrument reliably detected imposters who we knew were misrepresenting their age. For impostors, age did not correlate with performance on the instrument (r. = .077). Finally in Studies 3a, 3b, and 3c, the instrument remained robust with people from racial minority groups, low educational backgrounds, and those who had recently immigrated to the US. Thus, our instrument shows promise for verifying the age of online respondents, and, as we discuss, our approach of assessing “insider knowledge” holds great promise for verifying other identities within online studies.

Judges Behaving Badly: Judicial Misconduct and a Threat to Rights

“…They aim to provide researchers with high quality data by closely vetting and screening participants. This screening includes technical checks to ensure respondents do not send multiple responses from one IP address, monthly data quality checks, and similar to other platforms, researchers can flag participants that provide low quality responses (Hartman et. al.…”