Expert Clinician Certainty in Diagnosing Autism Spectrum Disorder in 16–30-Month-Olds: A Multi-site Trial Secondary Analysis

Klaiman, Cheryl; White, Steven J.; Richardson, Shana; McQueen, Emma; Walum, Hasse; Aoki, Christa; Smith, Christopher J.; Minjarez, Mendy; Bernier, Raphael; Pedapati, Ernest V.; Bishop, Somer; Ence, Whitney; Wainer, Allison; Moriuchi, Jennifer M.; Tay, Sew-Wah; Deng, Ye; Jones, Warren R.; Gillespie, Scott; Klin, Ami

doi:10.1007/s10803-022-05812-8

Cited by 11 publications

(18 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given this, together with the study's focus on diagnosing very young children 20,26 who can present with complex, evolving clinical profiles (eg, language, cognitive, or other challenges, co-occurring with or distinct from autism), expert clinicians were required to prospectively rate their level of certainty in each diagnosis. 22,23 Reference standard certainty is important because performance metrics of comparison tests depend inherently on the reliability and validity of the reference standard: if a participant's reference standard diagnosis is uncertain, then that participant's ground truth for comparison will also be uncertain. Certainty was rated prospectively with each expert clinician blinded to index test results.…”

Section: Reference Standard Diagnosismentioning

confidence: 99%

“…Certainty was rated prospectively with each expert clinician blinded to index test results. 22,23 Index Test Measurement of Social Visual Engagement Social visual engagement data were collected using 6 investigational eye-tracking devices, one at each clinical site. Device operators were staff technicians employed by each clinical site, trained for approximately 1 hour when each device was delivered, with no other required specialization or prior training.…”

Section: Reference Standard Diagnosismentioning

confidence: 99%

“…Expert clinicians were certain of reference standard diagnoses in 335 (70.5%) of the 475 participants, while there was uncertainty in 140 (29.5%). 22 The While chronological age was restricted to 16 to 30 months, children's demonstrated verbal abilities (in both diagnostic groups) varied from those delayed to levels approximating typical 4-month-old infants (eg, vocalizing with nonspecific syllable sounds 35 ) to those with advanced skills more common to 44-month-old children (3.7 years old) (ie, knowing several hundred or more words; producing competent phrase speech 25,35 ). For nonverbal abilities, the range varied similarly from skills approximating those of typical 8-month-old infants (eg, working on fine grasping; beginning to actively seek out-of-view objects 35 ) to those more common to typical 50-month-old children (4.2 years old) (independently performing self-help skills; knowing multiple numbers and amounts 35 ).…”

Section: Participantsmentioning

confidence: 99%

See 2 more Smart Citations

Eye-Tracking–Based Measurement of Social Visual Engagement Compared With Expert Clinical Diagnosis of Autism

Jones,

Klaiman,

Richardson

et al. 2023

JAMA

Self Cite

View full text Add to dashboard Cite

ImportanceIn the US, children with signs of autism often experience more than 1 year of delay before diagnosis and often experience longer delays if they are from racially, ethnically, or economically disadvantaged backgrounds. Most diagnoses are also received without use of standardized diagnostic instruments. To aid in early autism diagnosis, eye-tracking measurement of social visual engagement has shown potential as a performance-based biomarker.ObjectiveTo evaluate the performance of eye-tracking measurement of social visual engagement (index test) relative to expert clinical diagnosis in young children referred to specialty autism clinics.Design, Setting, and ParticipantsIn this study of 16- to 30-month-old children enrolled at 6 US specialty centers from April 2018 through May 2019, staff blind to clinical diagnoses used automated devices to measure eye-tracking–based social visual engagement. Expert clinical diagnoses were made using best practice standardized protocols by specialists blind to index test results. This study was completed in a 1-day protocol for each participant.Main Outcomes and MeasuresPrimary outcome measures were test sensitivity and specificity relative to expert clinical diagnosis. Secondary outcome measures were test correlations with expert clinical assessments of social disability, verbal ability, and nonverbal cognitive ability.ResultsEye-tracking measurement of social visual engagement was successful in 475 (95.2%) of the 499 enrolled children (mean [SD] age, 24.1 [4.4] months; 38 [8.0%] were Asian; 37 [7.8%], Black; 352 [74.1%], White; 44 [9.3%], other; and 68 [14.3%], Hispanic). By expert clinical diagnosis, 221 children (46.5%) had autism and 254 (53.5%) did not. In all children, measurement of social visual engagement had sensitivity of 71.0% (95% CI, 64.7% to 76.6%) and specificity of 80.7% (95% CI, 75.4% to 85.1%). In the subgroup of 335 children whose autism diagnosis was certain, sensitivity was 78.0% (95% CI, 70.7% to 83.9%) and specificity was 85.4% (95% CI, 79.5% to 89.8%). Eye-tracking test results correlated with expert clinical assessments of individual levels of social disability (r = −0.75 [95% CI, −0.79 to −0.71]), verbal ability (r = 0.65 [95% CI, 0.59 to 0.70]), and nonverbal cognitive ability (r = 0.65 [95% CI, 0.59 to 0.70]).Conclusions and RelevanceIn 16- to 30-month-old children referred to specialty clinics, eye-tracking–based measurement of social visual engagement was predictive of autism diagnoses by clinical experts. Further evaluation of this test’s role in early diagnosis and assessment of autism in routine specialty clinic practice is warranted.Trial RegistrationClinicalTrials.gov Identifier: NCT03469986

show abstract

Section: Reference Standard Diagnosismentioning

confidence: 99%

Section: Reference Standard Diagnosismentioning

confidence: 99%

Section: Participantsmentioning

confidence: 99%

See 1 more Smart Citation

Eye-Tracking–Based Measurement of Social Visual Engagement Compared With Expert Clinical Diagnosis of Autism

Jones,

Klaiman,

Richardson

et al. 2023

JAMA

Self Cite

View full text Add to dashboard Cite

show abstract

“…As with any tool, clinicians must be empowered to make an overall best‐estimate clinical diagnosis, using the process of standardized test administration, regardless of instrument‐derived ASD/non‐ASD classification. This is especially relevant in light of increasingly complex clinical referral populations, comprised of individuals with ASD with less prototypical ASD symptom presentations, as well as individuals without ASD with various psychiatric and medical diagnoses, all of whom may be likely to receive scores very near (and on either side of) the established cutoffs (Elias & Lord, 2022; Klaiman et al., 2022; Lord & Bishop, 2021; McDonnell et al., 2019).…”

mentioning

confidence: 99%

Commentary: Best practices and processes for assessment of autism spectrum disorder – the intended role of standardized diagnostic instruments

Bishop

Lord

2023

Child Psychology Psychiatry

View full text Add to dashboard Cite

Development of standardized diagnostic instruments has facilitated the systematic characterization of individuals with autism spectrum disorders (ASD) in clinical and research settings. However, overemphasis on scores from specific instruments has significantly detracted from the original purpose of these tools. Rather than provide a definitive “answer,” or even a confirmation of diagnosis, standardized diagnostic instruments were designed to aid clinicians in the process of gathering information about social communication, play, and repetitive and sensory behaviors relevant to diagnosis and treatment planning. Importantly, many autism diagnostic instruments are not validated for certain patient populations, including those with severe vision, hearing, motor, and/or cognitive impairments, and they cannot be administered via a translator. In addition, certain circumstances, such as the need to wear personal protective equipment (PPE), or behavioral factors (e.g., selective mutism) may interfere with standardized administration or scoring procedures, rendering scores invalid. Thus, understanding the uses and limitations of specific tools within specific clinical or research populations, as well as similarities or differences between these populations and the instrument validation samples, is paramount. Accordingly, payers and other systems must not mandate the use of specific tools in cases when their use would be inappropriate. To ensure equitable access to appropriate assessment and treatment services, it is imperative that diagnosticians be trained in best practice methods for the assessment of autism, including if, how, and when to appropriately employ standardized diagnostic instruments.

show abstract

“…Children with higher cognitive and language abilities and milder autism-related behaviors are more likely to be missed and often receive their diagnoses later than those with greater language delays and more pronounced autism-related behaviors. 2 The high degree of diagnostic uncertainty for young autistic children contributes to delays in access to supports and services. Receiving an autism diagnosis is a first step to qualifying for early behavioral therapies that have been shown to improve outcomes, including language, cognitive, social, and adaptive skills.…”

mentioning

confidence: 99%

Could an Eye-Tracking Test Aid Clinicians in Making an Autism Diagnosis?

Dawson

2023

JAMA

View full text Add to dashboard Cite

Expert Clinician Certainty in Diagnosing Autism Spectrum Disorder in 16–30-Month-Olds: A Multi-site Trial Secondary Analysis

Cited by 11 publications

References 32 publications

Eye-Tracking–Based Measurement of Social Visual Engagement Compared With Expert Clinical Diagnosis of Autism

Eye-Tracking–Based Measurement of Social Visual Engagement Compared With Expert Clinical Diagnosis of Autism

Commentary: Best practices and processes for assessment of autism spectrum disorder – the intended role of standardized diagnostic instruments

Could an Eye-Tracking Test Aid Clinicians in Making an Autism Diagnosis?

Contact Info

Product

Resources

About