Integration of audiovisual sensors and technologies in a smart room

Abstract

At the Technical University of Catalonia (UPC), a smart room has been equipped with 85 microphones and 8 cameras. This paper describes the setup of the sensors, gives an overview of the underlying hardware and software infrastructure and indicates possibilities for highand low-level multi-modal interaction. An example of usage of the information collected from the distributed sensor network is explained in detail: the system supports a group of students that have to solve a lab assignment related problem.

Depth image-based hand tracking in complex scene

“…Neumann et al [21] proposed a full-body tracking method in a special environment consisting of multiple cameras, called SmartRoom. Corazza et al [22] used visual hull data to track human body through a person-wise model which is updated by ICP (iterative closest point) measures.…”

Temporal contrast normalization and edge-preserved smoothing of temporal modulation structures of speech for robust speech recognition

“…The impulse response signals of the smart room depend on the SMDs (Neumann et al, 2007). The feature types and acoustic models are configured in the same manner as those used in the AURORA-2J experiments.…”

“…The reverberation speech is artificially developed by convolving clean speech (the same utterance as that used for the additive noise condition) with the impulse response of a smart room (with reverberant time around 650 ms) (Neumann et al, 2007). The temporal trajectory of the first-order coefficients is shown in Fig.…”

Temporal contrast normalization and edge-preserved smoothing on temporal modulation structure for robust speech recognition

“…For reverberant noise condition, we adopt the similar processing procedures. The reverberant speech is artificially generated by convolution between the clean speech (the same utterance as used for additive noise condition) with the impulse response of a smart-room (with RT of 650 ms) [7]. In both additive and reverberant noise conditions, the modulation spectrum is calculated using smoothed power spectrum of the cepstral time series.…”