Fusion of audio and video information for detecting speech events

Asano, Futoshi; Motomura, Y.; Nakamura, Satoshi

doi:10.1109/icif.2003.177472

Cited by 12 publications

(6 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The sources of additive noise in SHC include electric appliances commonly used in the household, such as radio, TV set, refrigerator, fume hood, vacuum cleaners, noise from the fan, air conditioning, etc., as well as cooking, children's noise, flushing of toilets, door slamming, people talking, outdoor traffic noise in the town, etc. At present, scientific workplaces worldwide are trying to solve the implementation of suitable platforms for voice control of automation in the real environment of intelligent buildings [80][81][82][83] with the use of suitable methods of additive noise suppression in the real SHC environment. The actual design and solution by the authors of additive noise suppression in the real SHC environment by the implementation of adaptive algorithms and Soft Computing methods in conformity with the block diagram in Fig.…”

Section: Voice Control Testingmentioning

confidence: 99%

Monitoring of the daily living activities in smart home care

Vaňuš

Belesova

Martínek

et al. 2017

Hum. Cent. Comput. Inf. Sci.

View full text Add to dashboard Cite

One of the key requirements for technological systems that are used to secure independent housing for seniors in their home environment is monitoring of daily living activities (ADL), their classification, and recognition of routine daily patterns and habits of seniors in Smart Home Care (SHC). To monitor daily living activities, the use of a temperature, CO 2 , humidity sensors, and microphones are described in experiments in this study. The first part of the paper describes the use of CO 2 concentration measurement for detecting and monitoring room´s occupancy in SHC. In second part focuses this paper on the proposal of an implementation of Artificial Neural Network based on the Levenberg-Marquardt algorithm (LMA) for the detection of human presence in a room of SHC with the use of predictive calculation of CO 2 concentrations from obtained measurements of temperature (indoor, outdoor) T i , T o and relative air humidity rH. Based on the long-term monitoring (1 month) of operational and technical functions (unregulated, uncontrolled) in an experimental Smart Home (SH), LMA was trained through the data picked up by the sensors of CO 2 , T and rH with the aim to indirectly predict CO 2 leading to the elimination of CO 2 sensor from the measurement process. Within the realized experiment, input parameters of the neuronal network and the number of neurons for LMA were optimized on the basis of calculated values of Root Mean Squared Error, the correlative coefficient (R) and the length of the measured training time ANN. With the use of the trained network ANN, we realized a strictly controlled short-term (11 h) experiment without the use of CO2 sensor. Experimental results verified high method accuracy (>95%) within the short-term and long-term experiments for learned ANN (1.6.2015ANN (1.6. -30.6.2015. For learned ANN (1.2.2014ANN (1.2. -27.2.2014) was verified worse method accuracy (>60%). The original contribution is a verification of a low-cost method for the detection of human presence in the real operating environment of SHC. In the third part of the paper is described the practical implementation of voice control of operating technical functions by the KNX technology in SHC by means of the in-house developed application HESTIA, intended for both the desktop system version and the mobile version of the Windows 10 operating system for mobile phones. The resultant application can be configured for any building equipped with the KNX bus system. Voice control implementation is an in-house solution, no third-party software is used here. Utilization of the voice communication application in SHC was proven on the experimental basis with the combination of measurement CO 2 for ADL monitoring in SHC.

show abstract

Section: Voice Control Testingmentioning

confidence: 99%

Monitoring of the daily living activities in smart home care

Vaňuš

Belesova

Martínek

et al. 2017

Hum. Cent. Comput. Inf. Sci.

View full text Add to dashboard Cite

show abstract

“…For sound localization, the MUSIC method [9] extended to a broadband signal with eigenvalue weighting [6] is used. We denote the input vector as x(ω, t) = [X 1 (ω, t), .…”

Section: Sound Localizationmentioning

confidence: 99%

“…The location and time information obtained from audio and video observation is fused by a Bayesian network so that the time and location of the speech event can be estimated [6,7].…”

Section: Introductionmentioning

confidence: 99%

Detection and Separation of Speech Event Using Audio and Video Information Fusion and Its Application to Robust Speech Interface

Asano

Yamamoto

Hara

et al. 2004

EURASIP J. Adv. Signal Process.

View full text Add to dashboard Cite

A method of detecting speech events in a multiple-sound-source condition using audio and video information is proposed. For detecting speech events, sound localization using a microphone array and human tracking by stereo vision is combined by a Bayesian network. From the inference results of the Bayesian network, information on the time and location of speech events can be known. The information on the detected speech events is then utilized in the robust speech interface. A maximum likelihood adaptive beamformer is employed as a preprocessor of the speech recognizer to separate the speech signal from environmental noise. The coefficients of the beamformer are kept updated based on the information of the speech events. The information on the speech events is also used by the speech recognizer for extracting the speech segment.

show abstract

“…Asano et al used AV integration for speech recognition, but their AV integration was limited to sound source localization [10]. Nakadai et al also reported that AV integration in the level of speaker localization and identification indirectly improved ASR in our robot audition system [11].…”

Section: Audio-visual Vadmentioning

confidence: 99%

An Improvement in Audio-Visual Voice Activity Detection for Automatic Speech Recognition

Yoshida

Nakadai

Okuno

2010

Trends in Applied Intelligent Systems

View full text Add to dashboard Cite

Abstract. Noise-robust Automatic Speech Recognition (ASR) is essential for robots which are expected to communicate with humans in a daily environment. In such an environment, Voice Activity Detection (VAD) strongly affects the performance of ASR because there are many acoustically and visually noises. In this paper, we improved Audio-Visual VAD for our two-layered audio visual integration framework for ASR by using hangover processing based on erosion and dilation. We implemented proposed method to our audio-visual speech recognition system for robot. Empirical results show the effectiveness of our proposed method in terms of VAD.

show abstract

Fusion of audio and video information for detecting speech events

Cited by 12 publications

References 11 publications

Monitoring of the daily living activities in smart home care

Monitoring of the daily living activities in smart home care

Detection and Separation of Speech Event Using Audio and Video Information Fusion and Its Application to Robust Speech Interface

An Improvement in Audio-Visual Voice Activity Detection for Automatic Speech Recognition

Contact Info

Product

Resources

About