Spatial Perception and Expectation: Factors in Acoustical Awareness for Mout Training

Abstract

Mixed Reality (MR), and its predecessor Virtual Reality (VR), has been primarily viewed as a visual science, with much less attention given to the other senses, despite clear evidence of their importance, especially audio. In fact, in military operation in urban environments, audio is often a more primary sense than vision, providing a soldier with an early warning system that needs to be honed and trained. Our research program, in contrast, treats the auditory sense as an equal to the visual. The consequences are MR experiences that have much greater impact than those in which audio is just an after thought. However, given the depth and breadth of graphics research, we are compelled to learn from this mature area. Thus, we are constantly striving to find results from graphics research that have useful analogies in the audio domain. Lessons learned from these analogies, especially as concern people's perception and expectations, are the focus of this paper. SUMMARYGraphics research is essentially about algorithms and a never-ending search for faster ways to render complex scenes. Faster graphics hardware, in the form of programmable graphics units (GPUs), have made it possible to achieve results that only a few years ago seemed unachievable on consumer grade machines. Nonetheless, the search for faster algorithms never ends, as user expectations rise to meet and exceed advances in hardware. To keep up with such demands, graphics researchers constantly take advantage of limitations in human perception and the expectations of viewers. The simplest example of gaining performance from human limitations is the use of clipping planes and object culling to reduce what must be rendered; after all, people cannot see behind their heads, and their peripheral vision, while acutely aware of motion, does not discern details well. As regards expectations, all computer graphics are based on the user's belief system, since what can be represented is so much more restricted than what can actually be discerned. Even simple cell animation plays on our expectations of continuity. What then can be learned from this that can lead to a less costly, more effective soundscape? This paper addresses the topic of expectation in audio, and how people's experiences play such a strong role in their audio belief systems.Despite attempts in software and hardware to deliver three dimensional audio, there has been little research on the aesthetic effects of sound design, and the influence of expectation in spatial perception and other more subjective measures (Begault, 1999). Our experiments indicate that expectations play a crucial role in our perception of sound localization, a key skill that soldiers use in identifying danger. In this context, expectations refer to the extent to which a sound is associated with a particular location (Cheung, 2002). In a pilot study conducted with 21 participants, sounds such as airplanes, helicopters, and lightning were perceived as being above head-level even when the sounds were played from speakers p...

The Evolution of a Framework for Mixed Reality Experiences

“…With all the compelling visual and haptic effects, one's hearing and training can provide a competitive edge, due to a heightened acoustical situational awareness (Hughes et al, 2004). You can't see or feel through walls, around corners or behind your head.…”

“…Other times we need to develop new scientific results, especially in the algorithmic area as in addressing realistic illumination and associated shading and shadowing properties in interactive time (Konttinen et al, 2005). Yet other times we must create new artistic conventions to deal with issues not easy solved by technology or science, e.g., taking advantage of people's expectations in audio landscapes (Hughes et al, 2004).…”