Several validated adult autism symptom screening tools exist; however, there are concerns about the validity of instruments in adults who self-identify and those who have an autism diagnosis. This study evaluated the psychometric properties of the RAADS-R and RAADS-14 across gender, autism diagnosis and autistic identity, and age among a sample of 839 adults. Participants included individuals who reported a prior diagnosis of autism, self-identified as autistic without a diagnosis, were exploring a diagnosis of autism, or neurotypical. Psychometric evaluation included confirmation of scale factor structure, and examination of item performance, including bias and discrimination, across the target sociodemographic characteristics. The RAADS-R and RAADS-14 were found to be psychometrically sound, unidimensional instruments in the present sample without systematic bias by age, diagnosis/identity, or gender. The use of dichotomous response options would improve measure parsimony. Close examination of extant item-level bias is also recommended in the development of new measures. Notably, few psychometric differences between diagnosed and self-identifying individuals were identified. There was a distinction, however, between diagnosed individuals and those exploring autistic identification. Four items representing the consequences of adverse sensory experiences and the hidden rulebook of social interaction were most informative for differentiating autistic individuals from neurotypical. Lay abstract Surveys used to screen for autism are being used much more by adults. Adults and clinicians want to be confident that the results from their surveys are accurate. If scores are not accurate, it is not possible to compare the scores of different groups of individuals. There are also concerns that these surveys do not accurately identify autistic individuals. This study explored the accuracy of two commonly used autism screening surveys for adults: the RAADS-R and the RAADS-14. The accuracy of these two screening tools was measured using a sample of 839 adults. Adults in the study were in one of the following categories: (1) diagnosed with autism, (2) adults who considered themselves to be autistic but had not been diagnosed, (3) adults who were unsure whether they were autistic, and (4) adults who did not consider themselves to be autistic and had not been diagnosed. The study found that the RAADS-R and the RAADS-14 are accurate. The study also found that a person’s age, gender, autism diagnosis, or whether an individual considered themselves to be autistic did not impact how they understood the survey. Survey accuracy could be improved by changing the number of question responses from four to two. Importantly, individuals with a clinical diagnosis of autism and those who considered themselves to be autistic responded to survey items in a very similar way. Individuals with autism diagnoses and those who were unsure whether they were autistic were more different in their responses. Four specific survey items related to sensory experiences and social interaction identified key differences between autistic and non-autistic adults.
