Challenges in quantifying multisensory integration: alternative criteria, models, and inverse effectiveness

Abstract

Single neuron studies provide one foundation for understanding many facets of multisensory integration. These studies have used a variety of criteria for identifying and quantifying multisensory integration. While a number of techniques have been used, there lacks an explicit discussion of the assumptions, criteria, and analytical methods traditionally used to define the principles of multisensory integration. This was not problematic when the field was small, but with rapid growth a number of alternative techniques and models have been introduced, each with its own criteria and sets of implicit assumptions to define and characterize what is thought to be the same phenomenon. The potential for misconception prompted this reexamination of traditional approaches in order to clarify their underlying assumptions and analytic techniques. The objective is to review the traditional quantitative methods advanced in the study of single-neuron physiology in order to appreciate the process of multisensory integration and its impact.

Sensory and Striatal Areas Integrate Auditory and Visual Signals into Behavioral Benefits during Motion Discrimination

“…These suppressive interactions cannot easily be explained by saturation or other nonlinearities in the BOLD response (Buxton et al, 2004), but rather suggest sublinear neural mechanisms of audiovisual integration and are consistent with previous fMRI results of object categorization and speech recognition in which only subadditive interactions at the level of the BOLD response were observed (Werner and Noppeney, 2010a,b;Lee and Noppeney, 2011). In the putamen and sensory areas, the operational mode of multisensory integration depended on the reliability or effectiveness of the sensory motion information, as expected under the classical principle of inverse effectiveness (Meredith and Stein, 1983;Stein and Meredith, 1993;Stanford and Stein, 2007;Stein and Stanford, 2008). Therefore, even though the interactions were always subadditive, they turned from response suppression for intact stimuli into a significant response enhancement for degraded stimuli.…”

“…In contrast, our study revealed subadditive integration profiles concurrently in both auditory and visual areas in the absence of parietal activations, suggesting integration mechanisms at the level of sensory motion processing. Indeed, recent studies have accumulated evidence for low-level multisensory integration in putatively unisensory areas (Schroeder and Foxe, 2002;van Atteveldt et al, 2004;Macaluso and Driver, 2005;Murray et al, 2005;Schroeder and Foxe, 2005;Bonath et al, 2007;Kayser et al, 2007;Lakatos et al, 2007;Lewis and Noppeney, 2010).…”

“…Therefore, even though the interactions were always subadditive, they turned from response suppression for intact stimuli into a significant response enhancement for degraded stimuli. Despite the fundamental role of inverse effectiveness in multisensory research (Stanford and Stein, 2007;Stein et al, 2009), its mechanism and functional role are still debated. Recent modeling work showed that inverse effectiveness may emerge naturally from divisive normalization as a simple mathematical operation (Ohshiro et al, 2011) in which the final response of a neuron is generated by dividing the activity of each neuron by the net activity of all neurons within a region.…”

Auditory Event-Related Response in Visual Cortex Modulates Subsequent Visual Responses in Humans

“…A clear effect of multisensory integration, i.e., cross-modal stimuli resulting in stronger responses than either auditory or visual stimuli alone (subadditive enhancement) (Stein et al, 2009), was found for the theta response only. This finding is in line with other studies that found effects of multisensory integration within the theta frequency band (e.g., Sakowitz et al, 2005).…”

Cerebral Correlates and Statistical Criteria of Cross-Modal Face and Voice Integration

“…There has been much discussion however around the pros and cons of the statistical criteria used to classify multisensory integration when comparing bimodal to unimodal conditions using fMRI (Beauchamp, 2005;Calvert, 2001;Goebel and van Atteveldt, 2009;Laurienti et al, 2005;Stein et al, 2009). The three main criteria used in fMRI research are: 1) the super-additive criteria, which requires the bimodal response to be greater than the sum of both unimodal responses; 2) the max criteria that requires the bimodal response to be greater than the largest unimodal response and; 3) the mean criteria requiring the bimodal response to be greater than the mean of the unimodal responses.…”