Human information processing and sensory modality: Cross-modal functions, information complexity, memory, and deficit.

Abstract

The relationship between sensory modality and information processing was examined. Available hypotheses concerning modality influence were not supported, but an interaction between modality and information complexity could be discerned. With simpler information (reaction signals and unidimensional discriminations) modalities are equivalent; contextual and parametric factors determine intermodal relations. With complex information (spatial or temporal patterns) specific modalities are adept with certain kinds of information, and what is received in nonadept modalities is recoded in the most adept. Generalizations about modality-information interactions depend on control of response factors. Implications of data and generalizations for theories .of memory, for measuring deficit in pathology, and for resolving the nativismempiricism controversy are discussed.

Strategies in cross-modality matching

“…These data were analyzed in a 2 (instructions) x 2 (delay) x 4 (condi- The results of the present exptriment are of interest in two respects. First, they replicate earlier studies in showing that cross-modality asymmetries can be obtained reliably, at least under some conditions (see Freides, 1974). Second, they demonstrate that subjects' early knowledge of the comparison modality can affect performance in specific ways.…”

“…While this model explains Connolly and Jones's findings and appears correct in emphasizing memory differences (Rose, Blank, & Bridger, 1972; Garvill & Molander, Note 1), its inadequacy as a general theory of intersensory integration has become clear. The predicted superiority of tasks in which vision is the comparison modality has been replicated (Cashdan, 1968;Freides, 1974; Jones Marteniuk & Rodney, 1979;Millar, 1972; Garvill & Molander, Note 2). But there are numerous studies reporting no asymmetries (Diewaert & Stelmach, 1977;Jones, 1973;Milner & Bryant, 1970;Newell, Shapiro, & Carlton, 1979;Rose et al, 1972) and asymmetries opposite to those predicted (Goodnow, 1971; Garvill & Molander, Note 1).…”

“…Attempts to explain this diversity of findings have concentrated almost entirely on stimulus differences. Explanations for the presence or direction of asymmetries have been based on whether or not the stimulus is spatial in nature (Pick, 1974), haptic or kinesthetic (Garvill & Molander,Note 1), high in informational complexity (Freides, 1974(Freides, , 1975 or presented in the context of a visible surround (Newell et al, 1979).…”

Interaction of Sight and Sound in the Perception and Experience of Musical Performance

“…In the video-only ratings of tempo variability, the notably larger effect size for Type of Video relative to Type of Time-warp (which represented the timing model to which the animation was time-warped and matched) suggests that participants used the simple amount of movement -rather than the pattern of timing of those movements -as a cue. This finding may be explained by the limited temporal resolution of the visual modality (e.g., Freides, 1974;Welch, DuttonHurt, & Warren, 1986), as well as the strong real-world association between the size of performers' movements and the amount of tempo and loudness variation.…”

“…Previous research has shown the temporal resolution of the auditory modality to be superior to that of the visual modality (e.g., Burr, Banks, & Morrone, 2009;Freides, 1974;Repp & Penel, 2002), resulting in superior auditory rhythm and beat perception (e.g., Grahn, 2012). However, previous research has also shown that visual kinematic information can influence the perceived duration of notes played on a marimba (Schutz & Kubovy, 2009;Schutz & Lipscomb, 2008), and that the sensitivity to rhythmic deviations can be modulated by point-light animations of a bouncing person (Su, 2014).…”