The Multisensory Attention Assessment Protocol (MAAP): Characterizing individual differences in multisensory attention skills in infants and children and relations with language and cognition.

Abstract

Multisensory attention skills provide a crucial foundation for early cognitive, social, and language development, yet there are no fine-grained, individual difference measures of these skills appropriate for preverbal children. The Multisensory Attention Assessment Protocol (MAAP) fills this need. In a single video-based protocol requiring no language skills, the MAAP assesses individual differences in three fundamental building blocks of attention to multisensory events—the duration of attention maintenance, the accuracy of intersensory (audiovisual) matching, and the speed of shifting—for both social and nonsocial events, in the context of high and low competing visual stimulation. In Experiment 1, 2- to 5-year-old children (N = 36) received the MAAP and assessments of language and cognitive functioning. In Experiment 2 the procedure was streamlined and presented to 12-month-olds (N = 48). Both infants and children showed high levels of attention maintenance to social and nonsocial events, impaired attention maintenance and speed of shifting when competing stimulation was high, and significant intersensory matching. Children showed longer maintenance, faster shifting, and less impairment from competing stimulation than infants. In 2- to 5-year-old children, duration and accuracy were inter-correlated, showed increases with age, and predicted cognitive and language functioning. The MAAP opens the door to assessing developmental pathways between early attention patterns to audiovisual events and language, cognitive, and social development.

Attentional abilities constrain language development: A cross‐syndrome infant/toddler study

“…This fits with the hypothesis that shifting one's focus of attention to objects being labelled strengthens correct word‐object associations and prunes incorrect ones, while focusing on the wrong object during labelling builds competing incorrect word‐object associations (Kucker, McMurray, & Samuelson, 2015). It fits with evidence that visual processing bootstraps word learning (Bahrick et al, 2018). It also fits with the data that support a broader hypothesis that the ability to disengage and shift attention facilitates the coordination of attention between the child and the caregiver (joint attention) (Hood, Willen, & Driver, 1998), which in turn feeds into the emergence of social skills, including language learning (Carpenter, Nagell, Tomasello, Butterworth, & Moore, 1998).…”

“…It is therefore possible that infants use eye gaze to infer what object or event is being referred to and labelled by their caregiver; certainly, infants are more likely to learn an object's label if the object being labelled dominates their visual field (Pereira, Smith, & Yu, 2014). Also, the ability to match auditory and visual information positively correlates with language ability, which suggests that visual processing bootstraps word learning (Bahrick, Todd, & Soska, 2018). Consequently, any irregularity in shifting attention may constrain how one interacts with, and thus learns from, the environment, with deleterious effects.…”

Remote Data Collection During a Pandemic: A New Approach for Assessing and Coding Multisensory Attention Skills in Infants and Young Children

“…Coding gaze in real time by trained observers is a widespread method for quantifying looking time. Observers, blind to the conditions of the study and unable to see the presentation of visual stimuli, estimate gaze direction and duration in real time while the participant views the stimuli (e.g., Lewkowicz, 1988;Bahrick et al, 2018a). This method is more time-efficient than frame-by-frame coding and requires little post processing of the data.…”

“…We found it relatively easy to convert parent questionnaires and assessments of children's language, social, and cognitive functioning, to this format. However, collecting looking time data for audiovisual tasks (i.e., tasks that track infant attention to multiple dynamic visual events in the presence of a soundtrack matching one of them) including the Multisensory Attention Assessment Protocol (MAAP; Bahrick et al, 2018a), posed significant challenges. For example, there are large individual differences in participants' home computer setups (e.g., differences in screen size, web camera quality, lighting, internet speed, etc.…”

“…For our purposes, we opted to use Gorilla Experiment Builder for the following reasons: (1) it provides excellent control of temporal parameters (e.g., trial onsets and offsets), (2) optimal video playback (e.g., little lagging, synchronous audio, and video playback), (3) an intuitive interface for building experiments so that they can be quickly deployed for remote data collection, and (4) can easily and efficiently be deployed in the home with minimal technology (e.g., experiments can be accessed through a web browser; Anwyl-Irvine et al, 2020a,b). In this paper, we detail our approach for collecting and coding looking time data remotely from our MAAP protocol-a three-screen (left, right, center displays) individual difference measure of three foundational attention skills (duration of looking, speed of shifting/disengaging, accuracy of audiovisual matching) to audiovisual social and non-social events (Bahrick et al, 2018a)using widely available software and hardware available on home computers. We then describe our approach for scoring data from the MAAP that have been collected in the home, using a newly developed platform for estimating gaze behavior from video recordings (OpenFace), as well as our development of a machine learning (ML) model to translate the estimates provided by OpenFace into meaningful looking time data (i.e., left, right, and center displays).…”